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Diis  report  consisting  of  a  user’s  guide  (Part  \)  and  back-up  data  (Part 
A),  was  developed  to  provide  reliability  program/ task  cost  guidelines  to 
DoD  program  reliability  managers  and  monitors.  The  primary  use  of  the 
guidelines  is  for  assistance  in  tailoring  task  provisions  of  MIL-STD-785 
and  MIL-STD-1543  (USAF)  as  applied  to  space  and  missile  systems.  Dis¬ 
played  data  includes  program/ task  cost  statistics  developed  as  a  function 
of  program/ system  phase  and  other  significant  characteristics.  The  study 
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SUMMARY 


This  report,  consisting  of  a  user's  guide  (Part  I)  and  back-up  data 
(Part  II),  was  developed  to  provide  reliability  program/task  cost 
guidelines  to  DoD  program  reliability  managers  and  monitors.  The 
primary  use  of  the  guidelines  is  for  assistance  in  tailoring  the 
task  provisions  of  K1L-STD-785  and  MIL-STD-1543  (USAF)  as  applied  to 
space  and  missile  systems. 

Data  developed  and  incorporated  in  the  guidelines  includes 
program/task  cost  statistics  derived  from  study  and  analysis  of 
thirteen  different  missile  and  space  programs.  In  addition,  results 
of  a  survey  on  the  impact  of  program/ system  characteristics  on 
program/task  costa  and  on  various  tailoring  options  are  included. 

Reliability  program/task  cost  statistics  are  presented  in  terms  of 
manhours,  percentage  of  engineering  budget  and  percentage  of 
reliability  program  budget.  The  percentage  figures  are  normalized 
to  annual  expenditures  to  provide  annual  effort  intensity  measures. 
Statistics  given  include  averages  and  measures  of  deviation  for 
various  program  types  and  phases.  Guidelines  are  provided  for  the 
application  of  these  statistics  and  associated  program/ system 
descriptive  data  to  the  problem  of  scoping  and  tailoring  program 
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reliability  efforts.  Tailoring  considerations,  based  on  the 
combined  effects  of  task  cost  and  task  effectiveness,  are  also 
included  as  part  of  the  guidelines  material.  Finally,  supplementary 
material  i.e.,  ground  rules,  definitions  and  descriptive  data,  is 
presented  to  properly  qualify  and  detail  the  study  results. 
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Failure  Reporting,  Analysis,  and  Corrective 
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Manmonths  per  Year;  a  Measure  of  Intensity  or 
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Procuring  Activity  (Including  Program/Project 
Offices) 
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BUDGET  Percent  of  Engineering  Budget:  A  measure  of  a 

program  expenditure  expressed  as  a  percentage 
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the  program 

Production  Reliability  Acceptance  Test 
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1.0 


INTRODUCTION 


A  NEED  FOR  RELIABILITY  COST  GUIDELINES 


Ideally,  the  reliability  task  provisions  of  MIL-STD-785  or 
MIL-STD-1543  (USAF)  could  be  tailored  to  a  particular  program  or 
system  without  critical  reference  to  task  costs.  Rationale  for  the 
tailoring  process  would  be  based  on  selecting  and  implementing  only 
those  tasks  to  the  extent  and  level  of  detail  that  made  sense  for  a 
particular  program  or  system.  The  features  of  a  particular 

I 

program/ system  like  development  phase,  or  system  development  status 
would  dictate  the  tailoring  process. 


Since  ideal  conditions  are  the  exception  rather  than  the  rule, 
reliability  managers  or  monitors  must  be  prepared  to  deal  with 
budget  constraints,  biases  and  other  factors  which  can  significantly 
affect  their  reliability  programs.  Each  of  these  factors, 
especially  that  of  budget,  must  be  dealt  with  properly  to  avoid 
either  short  sighted  planning  leading  ultimately  to  higher  costs  or 
to  avoid  inefficient  use  of  resources  which  yield  merely  an  illusion 
of  reliability  assurance. 


The  following  set  ol  reliability  program/task  cost  guidelines  have 
been  developed  to  assist  the  program  reliability  manager /monitor  in 
the  job  of  tailoring  a  reliability  program.  First,  the  guidelines 
provide  a  general  picture  of  reliability  program  and  task  costs  for 
a  typical  space  or  missile  system.  These  more  general  cost 
estimates  satisfy  the  two-fold  need  for  baseline  criteria  and  for  a 
standard  of  credibility.  Second,  the  guidelines  provide  cost  data, 
costing  methods,  and  task  effectiveness  criteria  useful  in 
conducting  trade  studies  in  support  of  tailoring  decisions. 

Together,  the  guidelines  provide  a  proven  basis  for  judging  the  cost 
and  effectiveness  impact  of  tailoring  decisions  for  either  a 
selected  task  or  a  total  reliability  program. 

The  following  figure  provides  a  simplified  flow  diagram  of  the  task 
tailoring  process.  Shown  are  the  process  essentials  and  how  these 
essentials  flow  and  interface  in  the  development  of  an  effective 
reliability  program. 

1.2  Guideline  Material  Arrangement 

Material  in  the  guidelines  is  grouped  and  sequenced  as  shown  in 
Table  1-1.  For  those  users  desiring  only  "broad  brush”  information, 
an  overview  is  provided  in  Section  2.0.  Section  3.0  (covering 


TABLE  1-1  -  GUIDELINE  MATERIAL  ARRANGEMENT 
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Shows  development  of  cost  estimation  regression  equations 


"step-by-step"  cost  estimation  procedures)  and  Section  4.0  (dealing 
with  tailoring  concepts)  are  for  specific  use  by  reliability  program 
managers  and  monitors.  Finally,  a  series  of  appendices  is  provided 
for  those  desiring  an  in-depth  view  of  the  underlying  data  base  and 
analysis  methods. 


2.0 


OVERVIEW 


2.1  Historical  Perspective 

It  is  not  intended  that  the  statistical  cost  data  presented  in  this 
Overview  Section  be  used  for  reliability  program  and  task  cost 
estimation  purposes.  The  data  have  not  been  "scrubbed”,  "purged"  or 
otherwise  altered  to  reflect  the  many  qualifications  and  nuances 
required  of  any  in-depth  analysis.  Presentation  of  the  data  is  made 
to  provide  the  guide  user  a  general  picture  of  what,  on  the  basis  of 
historical  cost  data  gathered  from  a  spectrum  of  space  and  missile 
programs,  constitutes  average  reliability  program  expenditures.  The 
cost  guideline  data  and  associated  application  procedures  for  use  in 
estimating  baseline  and  baseline  variant  reliability  program  and 
task  expenditures  are  presented  in  Section  3.0. 

Raw  cost  data  was  derived  from  a  study  of  reliability  program  and 
task  costs  on  thirteen  missile  and  space  system  programs.  These 
programs,  described  briefly  in  Table  2-1,  cover  the  major  phases 
(Concept  Phase  excluded)  of  development  and  production  and  range  in 
scope  from  large  missile  (ICBK)  to  small  spacecraft  (Satellite). 
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SMALL  SPACE  DEV/PROD  I  51  4.25  USAF  MIL-STD-785A 


Summaries  of  the  reliability  expenditures  on  the  above  programs,  for 
both  individual  and  average  cases,  are  shown  in  Tables  2-2  through 
2-7.  Expenditures  are  given  in  terms  of  marunonths  per  year  (MM/YR), 
percentage  of  engineering  budget  and  percentage  of  reliability 
budget  to  afford  easy  reference  to  several  common  standards.  Listed 
tasks  and  task  combinations  are  defined  in  Section  3.2. 

Table  2-2  presents,  on  an  individual  program  basis,  a  comparison 
between  engineering  and  reliability  program  MM/YR  expenditures.  The 
engineering  program  expenditures  are  ordered,  from  highest  (Program 
E)  to  lowest  (Program  L).  This  same  program  sequence  (E,  F,  H, 
G,...,L)  is  used  for  the  reliability  program  expenditures  to  give 
visibility  to  the  correlation  between  engineering  and  reliability 
efforts^  The  MM/YR  cost  measure  indicates  the  average  program 
intensity  level  over  the  interval  during  which  cost  statistics  were 
gathered. 

A  similar  comparison  between  engineering  and  reliability  program 
expenditures,  but  as  a  function  of  program  phase  and  program  type, 
is  provided  in  Table  2-3.  Here,  the  MM/YR  expenditures  reflect  the 
average  of  the  programs  fitting  the  category  descriptions.  Again, 
the  high  correlation  between  engineering  and  reliability  program 
efforts  is  evident.  Reliability  program  efforts  on  major  programs 
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(all  programs  except  the  two  small  spacecraft  programs)  range  from 
an  average  of  326  MM/YR  during  FSED  through  an  average  of  148  MM/YR 
during  the  PROD  phase  to  46  Ki-l/YR  for  the  VALID  phase. 

Small  space  programs  constitute  an  exception.  This  category  of 
program  is  discussed  in  more  detail  in  Section  3.0. 

Table  2-4  summarizes  the  same  reliability  program  expenditure  data 
as  presented  in  Tables  2-2  and  2-3,  but  in  terms  of  engineering 
budget  percentages.  While  considerable  variation  is  displayed  for 
the  individual  programs,  the  average  values,  with  the  exception  of 
that  for  small  space  programs,  show  a  remarkable  consistency  within 
the  4%  to  5%  level.  The  "tent  pole"  program,  Program  D,  contained  a 
part  requalification  activity  where  all  part  screening  costs  were 
charged  to  reliability. 

Major  program  reliability  task  expenditures  in  terms  of  percentage 
of  reliability  budget  are  presented  in  Table  2-3  as  a  function  of 
program  phase.  Similar  displays  as  a  function  of  program  type  are 
given  in  Table  2-6.  The  significance  of  the  distribution  of  effort 
within  a  reliability  program  is  discussed  in  Section  2.2. 
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TABLE  2-5 


MAJOR  PROGRAM  RELIABILITY  TASK  EXPENDITURES 
BY  PROGRAM  PHASE  -  %  OF  RELIABILITY  BUDGET 


13 


Z  OF  ANNUAL  RELIABILITY  PROGRAM  BUDGE' 


TABLE  2-6 


RELIABILITY  TASK  EXPENDITURES  BY 
PROGRAM  TYPE  -  %  OF  RELIABILITY  BUDGET 
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Ranked  comparisons  of  various  measures  of  engineering  and 
reliability  effort  expenditures,  including  those  displayed  in  the 
previous  tables,  are  given  in  Table  2-7.  Shown  for  each  of  thirteen 
programs  studied  are;  1)  measurement  period,  2)  total  engineering 
budget  for  the  measured  period,  3)  annual  engineering  budget,  4) 
total  reliability  budget  for  the  measured  period,  3)  annual 
reliability  budget,  and  6)  the  ratio  of  the  total  reliability  budget 
to  the  total  engineering  budget.  Adjacent  to  each  budget  measure  is 
the  ranking  of  the  measure  from  highest  (1)  to  lowest  (13). 

A  reasonably  high  degree  of  correlation  is  exhibited  between  the 
various  measure  pairs  with  the  exception  of  the  normalized  measure 
of  ratio  of  reliability  budget  to  engineering  budget  which 
neutralizes  the  effects  of  program  size.  This  measure  masks  some  of 
the  effects  from  program  characteristics,  such  as  complexity,  for 
the  same  reason  that  it  neutralizes  the  effects  of  program  size. 

The  correlated  measures  allow  options  in  the  selection  of  cost 
statistics  depending  on  such  factors  as  data  availability  and  the 
particular  object  of  the  cost  analysis. 
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2.2 


Reliability  Cost  Drivers 


Reliability  program  cost  drivers  can  be  considered  from  two 
viewpoints:  1)  high  cost  tasks  relative  to  the  total  reliability 
effort  and  2)  high  cost  total  reliability  efforts  as  a  result  of 
particular  program  or  system  characteristics  or  features. 

Referring  to  Table  2-5  of  Section  2.1  of  the  Overview,  a  ranked 
listing  of  the  top  six  tasks  by  relative  cost  and  as  a  function  of 
program  phase  is  given  in  Table  2-8  below. 

TABLE  2-8  -  RELIABILITY  TASK  COST  DRIVERS 


TASK 

VALIDATION 

FSED 

PRODUCTION 

PARTS 

47Z 

48Z 

34Z 

FRACAS/FRB 

7Z 

12Z 

15Z 

ENGR.  SUPPORT 

III 

71 

24Z 

FMECA 

4Z 

10Z 

OZ 

TESTING 

32 

62 

9Z 

ANALYSIS 

10Z 

6Z 

10Z 

TOTAL 

82Z 

69Z 

92Z 

These  cost  driver  tasks,  which  are  discussed  further  below,  absorb 
some  80-90%  of  the  total  reliability  budget. 


The  parts  task  is  the  dominant  task  for  all  phases,  ranging  from 
47%-48%  during  the  VALID  and  FSED  phases  to  34%  during  the  PROD 
phase.  Other  conspicuous  tasks  are  FRACAS  and  a  catch-all  task 
called  "engineering  support".  The  latter  task  refers  to  those 
miscellaneous  activities  conducted  by  non-reliability  engineering  in 
support  of  reliability  tasks.  These  activities,  separate  from  the 
parts,  SCA  or  tolerance  analysis  task  efforts,  are  done  in  support 
of  reliability  analysis,  FMECA's,  subcontractor  control,  etc. 

Reliability  testing  tasks,  although  potentially  high  cost,  did  not 
turn  out  to  be  cost  drivers.  There  are  two  reasons  for  this 
outcome:  1)  the  more  or  less  standard  practice,  where  possible,  of 
"piggybacking"  reliability  tests  on  tests  designed  for  other 
purposes  and  2)  contractor  costing  practices  of  charging  strict 
reliability  test  costs,  including  test  articles,  instrumentation, 
chamber  hardware,  etc.  to  either  engineering  or  manufacturing 
budgets.  What  remains  as  reliability  charges  are  the  costs 
associated  with  defining  reliability  test  requirements,  the  related 
planning  for  implementing  these  requirements  and  the  monitoring  of 


test  results 


Another  item  of  note  is  that  total  reliability  program  costs  were 
approximately  4%  of  the  engineering  budget  for  both  the  FSED  and 
PROD  phases.  Here  it  is  seen  that  while  the  F1IECA  task  drops  to 
zero  during  proauction,  the  FRACAS  actively  increases  by  some  50%. 
Other  tasks,  including  what  is  labeled  as  “modeling,  allocation  and 
prediction",  but  which  include  all  analysis  efforts,  do  not  show 
significant  shifts.  The  analysis  efforts  phase  from  a  primarily 
predictive  mode  to  an  assessment  (test  data  evaluation)  mode. 

Reliability  cost  drivers  noted  above  are  in  terms  of  the  relative 
cost  per  individual  task.  Cost  drivers  can  also  be  viewed  in  terms 
of  particular  program/ system  characteristics  or  features  which 
significantly  impact  reliability  program/task  costs.  Results  of  a 
survey  and  an  analysis  of  cost  data  on  this  subject  are  summarized 
below. 

Questionnaires  were  sent  to  a  number  of  reliability  program  manager 
and  lead  engineers  asking  them  first  to  indicate  those  program  or 
system  characteristics  which  significantly  impact  (either  plus  or 
minus)  individual  task  costs.  Responses  to  this  particular  question 
are  summarized  in  Table  2-9,  “Majority  Response  Matrix".  Indicated 
by  an  “X"  are  those  task  cost  vs  characteristic  relationships  which 
a  majority  of  respondents  deemed  cost  significant.  As  may  be  noted, 
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a  majority  of  respondents  felt  that  program  phase  vas  a  significant 
factor  in  all  listed  tasks.  Other  characteristics  which  were  felt 
to  have  broad  impact  were  hardware /software  development  status, 
complexity  in  terms  of  number  of  parts  and  reliability  criticality. 

A  second  item  on  the  questionnaire  was  a  request  to  give  a  ranked 
listing  of  the  program  or  system  characteristics  as  regards  to  cost 
impact.  Results  of  the  respondents  replies  are  summarized  in  Table 
2-10.  Reasonable  correlation  is  found  between  the  priority  listing 
of  Table  2-10  and  the  majority  response  matrix  of  Table  2-9. 

A  comparison  between  the  survey  responses  and  the  actual  cost  data 
from  the  programs  shows  agreement  that  program  phase  is  the  single 
most  significant  program  or  system  characteristic  in  regard  to  cost 
impact.  Other  characteristics,  such  as  hardware/ software  status  and 
number  of  parts  (complexity),  do  have  impact  on  reliability  program 
costs,  but  are  masked  when  dealing  with  the  percentage  of 
engineering  budget  as  a  measure  of  reliability  effort.  Reference  to 
engineering  or  reliability  program  manmonth  per  year  expenditures 
and  program  part  count  (or  number  of  part  types)  show  a  correlation 
between  complexity  and  cost. 
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TABLE  2-10  -  RANKED  COST  IMPACT  LIST  -  PROCRAM/SYSTEM  CHARACTERISTICS 


PROGRAM/SYSTEM  CHARACTERISTIC 


CONTRACT  PHASE 

NUMERICAL  REQUIREMENT  LEVEL 

NO.  OF  PARTS 

CONFIGURATION 

HARDWARE/SOFTWARE  STATUS 

MISSION  DURATION 

MISSION  ENVIRONMENT 

RELIABILITY  CRITICALITY 

NO.  OF  PART  TYPES  (OR  SUBSYSTEMS) 

SCHEDULE 

CONTRACTOR  ROLE 

PRODUCTION  QUANTITY 

UNIT  COST 

STORAGE  LIFE  REQUIREMENT 
MAINTENANCE  CONCEPT 
%  FAILURE  RATE  SUBCONTRACTED 
SERVICE  LIFE  REQUIREMENT 


RATING:  123456789  10 


POINTS:  10  98765432  1 


SCORE 


2.3 


Reliability  Program  Cost  Effectiveness 


Actual  measurement  of  reliability  program  effectiveness  and  the 
isolation  of  causes  for  lack  of  effectiveness  is  an  extremely 
difficult,  if  not  impossible,  task.  Of  the  thirteen  programs 
surveyed,  five  have  progressed  to  phases  where  measurement  of 
achieved  reliability  is  possible.  In  each  of  the  five  cases,  the 
required  reliability  level  has  been  achieved  or  exceeded  (See  Table 
A- 9  of  Appendix  A)  and  in  each  of  the  programs  certain  reliability 
problems  were  encountered  and  solved.  It  is  the  latter  activity, 
the  discovery  and  solution  of  reliability  problems,  which  has  the 
real  bearing  on  program  effectiveness.  A  well  planned  and  funded 
reliability  effort  can  prevent  or  ferret  out  problems  in  a  manner 
which  minimizes  program  costs.  Less  efficient  efforts  might 
ultimately  result  in  the  same  achieved  levels  of  reliability,  but  at 
much  greater  program  expenditures  and  possibly  with  delay  in 
achieving  operational  status. 

Further  discussions  on  reliability  cost  effectiveness  are  contained 
in  Section  4.0  -  RELIABILITY  PROGRAM  TAILORING.  This  section 
contains  both  a  general  treatment  of  the  tailoring  topic  such  as 
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considerations  of  program  effectiveness,  time  phasing,  mechanics, 
etc.  and  a  detailed  treatment  of  tailoring  decisions  on  a 
task-by-task  basis. 


3.0 


RELIABILITY  PROGRAM  AND  TASK  COST  ESTIMATION 


While  considerable  effort  has  gone  into  the  arrangement  and  display 
of  reliability  program/task  cost  data  and  associated  tailoring 
considerations,  maximum  use  of  this  guide  requires  effort  and 
judgement  on  the  part  of  the  user.  The  very  nature  of  tailoring 
implies  a  departure  from  the  general  or  average  case,  a 
consideration  of  the  unique  features  of  a  program/ system  and  an 
appreciation  of  the  dynamics  of  system  development.  Therefore, 
guide  users  are  urged  to  bring  their  own  specialized  knowledge  into 
play  along  with  the  information  in  the  guide  in  order  to  achieve  the 
most  effective  results. 

3.1  Guideline  Applicability 

Historical  cost  data,  used  in  developing  the  cost  guidelines, 
reflect  many  underlying  factors  or  conditions.  They  reflect  a 
contractor's  response,  including  any  tailoring,  budgetary  or 
management  influences,  to  program  and  system  requirements  as 
specified  in  the  statement  of  work  and  referenced  standards.  They 
also  reflect  the  type  of  system  under  development,  the  structure  of 
the  development  process,  the  relationship  existing  between  the 
contractor  and  the  customer  and  other  similar  factors.  Finally, 
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they  reflect,  to  some  degree,  the  particular  charging  or  costing 
practices  of  the  contractor  and  the  contractor's  approach  to 
reliability  program  implementation. 

Some  of  the  above  factors  have  been  formalized  as  program  and  system 
"characteristics"  and  their  effects  studied  and  evaluated.  Other 
factors  were  not  addressed,  either  because  they  were  not  measurable 
or  because  there  was  no  experience  base  available  for  their 
evaluation.  This  latter  category  of  factors,  those  factors  not 
measured  or  not  reflected  in  the  experience  base,  place  some 
limitations  on  the  applicability  of  the  data.  These  applicability 
limits  are  summarized  in  Table  3-1  and  are  discussed  in  more  detail 
in  the  following  paragraphs. 

3.1.1  System  Type 

Guide  data  is  specifically  applicable  to  military  missile  and  space 
systems  ranging  in  complexity  from  large  missile  (1CBM)  to  small 
space  (satellite).  It  is  also  applicable,  with  qualification,  to 
other  military  systems  and  to  NASA  unmanned  space  systems. 
Applicability  to  other  military  systems  depends  on  the  degree  of 
correspondence  of  their  development  process  with  that  of  missile  and 
space  systems.  NASA  reliability  programs  for  unmanned  space  systems 
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are  very  close  to  military  missile  and  space  programs.  Cost  driver 
tasks  such  as  parts,  FRACAS  and  FMECA  are  very  similar. 

3.1.2  Contractor  Level 

Reliability  program  and  task  costs  presented  are  based  on  those 
experienced  by  a  single  contractor  performing  at  the  system  or 
associate  contractor  level.  With  some  exceptions,  notably  in  areas 
relating  to  integration  of  activities,  the  results  are  also  believed 
applicable  to  the  major  subcontractor  level.  Cost  biases  due  to 
contractor  approach  to  reliability  or  to  particular  contractor 
charging  practices  are  not  believed  significant.  The  influence  of 
customer  program  and  system  requirements  would  appear  to  be  a  much 
more  dominant  factor  than  that  of  contractor  approach.  Any 
contractor  charging  biases  were  minimized  by  attempting  to  account 
for  all  reliability  task  costs  regardless  of  the  performing 
organization.  A  significant  advantage  of  limiting  the  study  to  a 
single  contractor  was  the  ability  to  perform  detailed  investigations 
of  all  data  base  entries.  These  investigations  were  accomplished  by 
means  of  follow-up  interviews  with  reliability  program  managers  and 
lead  engineers  associated  with  each  surveyed  program. 


3.1.3  Development  Program  Structure 

Eleven  of  the  programs  in  the  thirteen  program  data  set  generally 
followed  structured  development  phases  as  defined  in  MIL-STD-785. 
Exceptions  were  the  two  small  spacecraft  programs  which  were 
developed  and  produced  under  a  single  contract  phase.  The 
structured  phases  as  defined  in  MIL-STD-785  are  listed  below: 

(1)  Conceptual  (CONCEPT)  Phase:  The  identification  and 
exploration  of  alternative  solutions  or  solution 
concepts  to  satisfy  a  validated  need. 

(2)  Demonstration  and  Validation  (VALID)  Phase:  The 
period  when  selected  candidate  solutions  are  refined 
through  extensive  study  and  analyses;  hardware 
development,  if  appropriate;  test;  and  evaluations. 

(3)  Full-Scale  Engineering  Development  (FSED)  Phase: 

The  period  when  the  system  and  the  principal  items 
necessary  for  its  support  are  designed,  fabricated, 
tested  and  evaluated. 

(4)  Production  (PROD)  Phase:  The  period  from  production 
approval  until  the  last  system  is  delivered  and 
accepted. 


As  noted  in  the  Overview,  the  Concept  Phase  has  not  been  addressed 
in  the  guide  because  of  the  lack  of  specific  cost  data  associated 
with  this  phase  of  acquisition.  As  a  normal  practice,  reliability 
costs  during  the  concept  phase  are  small  and  recorded  integrally 
with  other  engineering  charges.  These  efforts,  consisting  mainly  of 
feasibility  analyses  and  trade  studies,  are  predominantly  part  time 
efforts  and  would  generally  be  less  than  or  at  most  equal  to  the 
percentage  of  engineering  effort  expended  in  the  validation  phase. 

3.1.4  Reliability  Program  Structure 

Reliability  program  structure  is  important  both  to  understanding  the 
historical  cost  data  and  to  applying  the  data  to  new  programs.  It 
serves  as  a  standard  for  defining  the  categories  of  reliability 
effort  and  for  judging  the  scope  of  the  individual  reliability 
tasks. 

DoD  Directive  3000.40,  "Reliability  and  Maintainability",  and  the 
Air  Force  implementing  policy,  AF  Regulation  800-18,  "Air  Force 
Reliability  and  Maintainability  Program",  address  the  need  to 
develop  reliability  programs  with  a  balanced  mix  of  tailored  design 
engineering  and  management  accountant  tasks  for  each  program  phase. 
Referenced  lower  tier  standards,  MIL-STD-785,  "Reliability  Program 
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for  Systems  and  Equipment  Development  and  Production”,  and 
H1L-STD-1 543 ,  "Reliability  Program  Requirements  for  Space  and 
Missile  Systems”,  list  the  basic  task  options  to  be  considered  in 
the  development  of  an  acceptable  program. 

Twelve  of  the  thirteen  programs  included  in  the  cost  data  base  were 
conducted  within  the  framework  of  one  or  the  other  of  the  two 
referenced  lower  tier  standards.  The  one  exception,  an  Army  missile 
program,  was  conducted  under  QR-800-M  which  is  similar  in  structure 
and  scope  to  MIL-STD-785.  A  summary  and  comparison  of  the 
reliability  tasks  contained  in  MIL-STD-7 85B  and  -1543  (Notice  2)  is 
provided  in  Table  3-2. 

Examination  of  Table  3-2  shows  more  or  less  general  agreement 
between  the  tasks  of  MIL-STD-785B  and  MIL-STD-1543 ,  Notice  2. 
Exceptions  are  Sneak  Circuit  analysis.  Environmental  Stress 
Screening  and  Monitor/Control  Subcontractor  and  Supplier  tasks 
contained  in  -785B  and  without  counterpart  in  -1543.  From  a 
compliance  viewpoint,  the  tasks  of  -1543  are  generally  more  specific 
than  those  of  -785B,  calling  for  proportionately  greater  detail  in 
tailoring  decisions. 


3.2 


Cost  Data  Development 


TABLE  3-2  -  TASK  COMPARISON  -  MIL-STD-785B  VS 

MIL-STD-1543,  NOTICE  2 


MIL-STD-785B 

MIL-STD-1543,  NOTICE  2 

NO. 

TASK  TITLE 

CORRESPONDING  PARAGRAPH 

101 

Reliability  Program  Plan 

4.0  (Implicit) 

102 

Monitor /Control  Sub.  &  Suppl. 

No  Explicit  Task 

103 

Program  Reviews 

4.3  (Also  Ref.  MIL-STD-1521) 

104 

Failure  Reporting  Etc  (FRACAS) 

5.5 

105 

Failure  Review  Board 

5.5.3 

201 

Reliability  Modeling 

5.4.1,  5.6 

202 

Reliability  Allocations 

5.4.1,  4.1,  5.6 

203 

Reliability  Predictions 

5.4.2,  5.3,  5.6 

204 

Failure  Modes,  Etc  (fMECA) 

5.2 

205 

Sneak  Circuit  Analysis  (SCA) 

No  Explicit  Task 

206 

Elec.  Parts/Clrc.  Tol.  Anal. 

4.6 

207 

Parts  Program 

Prog  Tailored, 

AF  SAMS0-STD- 7  3-2-C 

208 

Reliability  Critical  Items 

4.4 

209 

Effects  of  Funct.  Test  Etc. 

5.8,  5.7 

301 

Env.  Stress  Screen  (ESS) 

No  Explicit  Task 

302 

Rel.  Develop/Gwth.  Test 

4.5.1,  4.2 

303 

Rel.  Qual.  Test 

4.5.1,  4.2 

304 

Prod.  Rel.  Accpt.  Test 

4.2 

3.2.1  Task  Cost  Categories 

For  purposes  of  this  guide,  the  reliability  tasks  of  MIL-STD-7 85B 
are  employed  as  "standards"  insofar  as  they  are  generally 
descriptive  of  categories  of  reliability  efforts.  Their  use  in 
association  vith  defining  task  costs  is  not  precise,  however,  and 
this  fact  should  be  taken  into  account  in  developing  cost  estimates 
and  tailoring  criteria. 

In  interpreting  the  task  cost  data  presented  in  the  guide,  two 
sources  of  deviation  from  these  standards  should  be  noted.  In  the 
thirteen  program  data  set  only  Program  C,  a  validation  phase  effort, 
reflects  a  reliability  program  in  accordance  with  the  provisions  of 
MIL-STD-7 85B .  The  majority  of  the  remaining  programs  followed  the 
provisions  of  either  MIL-STD-785A  or  MIL-STD-1 543 .  Second,  the  cost 
accounting  practices  on  these  programs  did  not  follow  work  breakdown 
structures  in  exact  correspondence  with  the  specified  program 
reliability  tasks.  Often,  costs  were  recorded  in  combinations  of 
tasks  rather  than  individually. 

Table  3-3  provides  a  comparison  between  the  -785B  tasks  and  the 
tasks  used  in  the  study  to  accumulate  costs.  The  right  hand  column 
of  the  table  shows  three  cases  of  task  combination  in  regard  to  task 
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TABLE  3-3 


TASK  COMPARISON  MIL-STD-785B  VS  GUIDE 


MIL-STD-785B 

|  GUIDE 

NO. 

TASK  TITLE 

NO. 

TASK  TITLE 

101 

Reliability  Program  Plan 

1 

Same 

102 

Monitor/Cont.  Subcontractor 

n 

Same 

103 

Program  Reviews 

Same 

104 

Failure  Reporting,  Etc.  (FRACAS) 

■ 

FRACAS 

105 

Failure  Review  Board 

201 

Reliability  Modeling 

202 

Reliability  Allocations 

5 

►Analysis  (Also  Includes 

Assessments) 

203 

Reliability  Predictions 

204 

Failure  Modes,  Etc.  (FMECA) 

6 

Same 

205 

Sneak  Circuit  Analysis  (SCA) 

B 

Same 

206 

Elec  Parts/Circ.  Tol.  Anal. 

B 

Same 

207 

Parts  Program 

B 

Same 

208 

Reliability  Critical  Items 

10 

Same 

209 

Effects  of  Funct.  Test,  Etc. 

11 

Same 

301 

Env.  Stress  Screening  (ESS) 

302 

Rel.  Develop/Growth  Test 

12 

’  Reliability  Testing 

303 

Rel.  Qual.  Test 

304 

Prod.  Rel.  Accpt.  Test 

cost  reporting.  In  each  case,  the  combination  includes  only  those 
tasks  of  like  nature;  e.g.,  FRACAS  and  FRB.  The  combined  task 
titled  "analysis"  includes  not  only  modeling  allocation  and 
prediction,  but  also  the  added  activity  of  "assessment"  which  refers 
to  evaluation  by  means  of  actual  test  or  operational  data. 
Reliability  testing,  as  noted  earlier,  is  defined  to  include  only 
that  effort  associated  with  the  development  of  reliability  test 
requirements,  procedures,  evaluation  criteria,  etc.  It  does  not 
include  costs  associated  with  test  articles,  chambers, 
instrumentation,  test  technicians,  etc.  These  latter  costs  are 
recorded  against  either  Engineering  or  Manufacturing  depending  on 
the  test  objective.  For  example,  environmental  stress  screening  is 
considered  as  a  part  of  the  normal  manufacturing  process  resulting 
in  the  accounting  practice  of  charging  the  test  conduct  costs  to 
Manufacturing. 

With  the  exceptions  noted  above  on  task  combinations  and  task 
qualifications,  the  -785B  task  descriptions  can  be  employed  to 
approximate  the  scope  and  content  of  the  "mainline"  reliability 
tasks.  Task  descriptions,  abstracted  from  -785B,  are  provided  in 
Appendix  C  for  guide  user  convenience. 


Occasionally  it  is  useful  or  required  that  reliability  tasks  be 
listed  in  accordance  with  the  primary  performing  or  coordinating 
function.  The  listing  is  also  useful  in  isolating  and  estimating 
reliability  program  costs  when  these  costs  are  accumulated  at 
several  centers.  One  categorization,  developed  from  the  experience 
of  a  system/associate  level  contractor,  is  shovn  in  Table  3-4. 

3.2.2  Cost  Measures  and  Their  Development 

Two  primary  cost  measures  are  used  in  the  guide:  manmonths  per  year 

i 

and  percentage  of  a  defined  budget.  The  manmonth  per  year  measure, 
used  both  to  describe  intensity  levels  of  total  engineering  and 
reliability  efforts,  provides  an  "absolute”  standard  and  can  be 
scaled  to  fit  a  program  phase  of  any  given  duration.  The  percentage 
measure  provides  a  useful  relative  standard  both  for  scoping -total 
program  efforts  and  for  scoping  individual  tasks. 

Additionally,  the  percentage  figure  serves  as  a  normalized  measure 
discounting  for  the  factor  of  program  size.  Thus,  a  rule  of  thumb 
for  a  typical  FSED  reliability  program  might  be  42  of  the  total 
engineering  manmonths  per  year  or  42  of  the  total  engineering  budget 
for  the  entire  FSED  phase  (under  the  assumption  of  uniform  effort 
intensity).  In  a  similar  manner,  costs  of  individual  reliability 


TABLE  3-4 


TASK  CATEGORIES 


MIL- STD- 7 85 B 
TASK  NO. 

CATEGORY 

DESCRIPTION 

101 

Task  normally  performed  or  coordinated 

102 

by  the  reliability  project  organization. 

103 

104 

105 

201 

I 

202 

203 

204 

208 

290 

301 

302 

303 

304 

207 

II 

PMP  control  task  normally  performed  by 
a  technical  staff  group  assigned  to 
project. 

205 

206 

III 

Tasks  other  than  PMP  normally  performed 
by  project  or  supporting  technical  staff 
groups . 

(OTHER) 

General  engineering  support  given  to  the 
project  reliability  group  in  performance 
of  Category  I  tasks. 
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tasks  or  task  combinations  could  be  expressed  as  a  percentage  of  the 
reliability  program  budget. 

Basic  cost  data,  developed  for  use  in  the  guidelines,  were  derived 
from  recorded  or  planned  "manmonth"  expenditures  maintained  on  each 
missi le/space  program  surveyed.  For  each  program,  expenditures  were 
accumulated  for  a  defined  interval  which  varied  between  a  minimum  of 
27  months  to  a  maximum  of  75  months.  For  most  cases,  these  sampling 
intervals  were  chosen  either  to  correspond  to  or  lie  within  a  given 
program  phase. 

For  each  program  within  a  defined  program  phase  or  sampling 
interval,  manmonth  expenditure  data  were  grouped  within  three 
categories:  1)  engineering,  2)  reliability  and  3)  specific 
reliability  tasks  as  defined  subdivisions  of  categories  1)  or  2)  or 
both.  Interval  and  annual  manmonth  expenditures  as  well  as 
normalized  percentage  expenditures  were  developed  for  the  categories 
defined  above. 

3.3  Cost  Estimating  Procedure 

This  section  presents  the  cost  data  and  procedures  for  estimating 
total  reliability  program  and  individual  reliability  task  costs  for 
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missile  and  space  systems.  Minimum  information  required  for  use  of 
the  data  aud  procedures  are  definitions  of  the  system  type  (major 
acquisition  or  small  space)  and  the  phase(s)  of  development  (VALID, 
FSED,  PROD). 

Table  3.5  outlines  the  two  categories  of  estimation  methods 
presented.  Method  I  is  for  use  with  small  space  programs  and  major 
acquisition  systems  which  are  minimally  defined;  i.e.,  information 
on  system  characteristics  other  than  program  type  and  development 
phase  are  not  available.  Method  II  is  exclusively  for  well  defined 
major  acquisition  programs  and  systems  where  information  on  other 
characteristics  such  as  complexity,  configuration  and  hardware 
status  are  available. 

Both  Methods  I  and  II  also  use  an  estimate  of  the  program 
engineering  budget  to  develop  specific  manmonth  cost  data  for  the 
various  reliability  tasks  and  total  programs.  If  available,  the 
program  engineering  budget  can  be  considered  another  of  the  user 
supplied  system  characteristics. 

When  the  program  engineering  budget  is  not  available,  or  only 
generally  known  in  terms  of  high  or  low  boundary  conditions, 
guidelines  are  provided  to  aid  in  forming  an  initial  estimate. 
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Both  cost  estimating  methods  have  the  same  basic  approach: 
reliability  task  costs  are  estimated  independently  in  terms  of 
percentage  of  engineering  budget,  and  are  then  combined  to  form  the 
total  reliability  program  costs.  The  program  engineering  budget 
estimate  is  then  used  to  translate  these  values  into  specific 
manmonths  per  year  cost  measures. 

The  above  describes  two  of  the  cost  measures  produced  by  the  two 
methods.  The  complete  set  includes: 

•  Total  Reliability  Program  Cost  -  MM/Year 

•  Total  Reliability  Program  Cost  -  1  of  Engr.  Budget 

•  Reliability  Task  Costs  -  MM/Year 

•  Reliability  Task  Costs  -  Z  of  Engr.  Budget 

•  Reliability  Task  Costs  -  %  of  Rel.  Budget 

Each  method  is  described  by  step-by-step  procedures  which  include 
sample  work  sheets  and  example  problems.  Data  tables  and  associated 
information  appropriate  to  the  use  of  each  method  are  collected  in 
Sections  3.3.2  and  3.3.3. 

It  may  be  noted  that  derived  reliability  cost  values  differ  from 
their  counterparts  reported  in  Section  2.0  (Overview).  These 
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deviations  can  be  traced  to  differences  in  objectives  and  data 
qualification.  In  the  Overview,  the  intent  was  to  show,  from  a 
historical  perspective,  the  costs  incurred  on  a  selected  collection 
of  missile  and  space  programs.  In  satisfying  this  objective, 
summary  statistics  were  developed  from  the  cost  data  as  reported  and 
v/ithout  detailed  qualification. 

In  contrast  to  providing  historical  perspective,  the  objective  of 
this  section  is  to  develop  procedures  and  data  for  estimating  the 
reliability  program  and  task  costs  associated  with  the  tailored 
application  of  MIL-STD-785B  tasks  to  missile  and  space  programs.  As 
a  result,  the  raw  cost  data  has  been  "scrubbed"  to  eliminate  data 
items  not  representative  of  a  standard  785B  application.  Host 
often,  the  eliminated  data  points  reflected  a  tailoring  beyond  -785B 
standards.  The  cost  estimating  data  charts  used  herein  are  based  on 
this  modified  data  set  snd  reflect  the  standard  application  of  the 
respective  MIL-STD-785B  reliability  tasks.  Tailoring  of  the 
resultant  costs  is  treated  in  Section  4.0.  Details  of  the 
respective  data  bases  are  reported  in  Appendix  A. 


3.3.1 


Engineering  Budget  Estimation 


A  prerequisite  data  item  for  both  cost  estimation  procedures  is  the 
annual  program  engineering  budget.  If  available,  an  initial  program 
estimate  should  be  used  for  the  system  under  consideration.  When  no 
estimate  is  available,  one  can  be  developed  from  the  the  data  in 
Table  3.6.  This  table  shows  the  average  program  expenditures  in 
Mti/year  for  Small  Space  systems  and  for  Major  Acquisition  systems, 
with  the  latter  category  segregated  by  development  phase  (VALID, 
FSED,  PROD).  If  no  knowledge  of  system  complexity  or  of  the 
proposed  program  budget  is  available,  it  is  recommended  that  an 
average  program  budget  be  used  in  the  cost  estimation  procedures. 

In  some  cases  because  of  system  complexity,  program  size,  or  other 
general  system  characteristics,  this  average  budget  may  not  be 
considered  valid.  For  these  cases,  budget  ranges  are  presented  to 
aid  in  estimating  an  appropriate  program  budget. 

The  following  general  guidelines  developed  from  observations  on  the 
data  are  provided  as  additional  aids: 

•  The  major  influence  on  program  budgets  was  found  to  be 
system  complexity  *  the  more  complex  systems  had  the 
higher  costs. 
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•  A  secondary  influence  was  found  to  be  the  procurement 
schedule  -  accelerated  schedules  had  higher  annual 
engineering  budgets. 

•  Another  secondary  influence  observed  was  program  size. 
Generally  high  production  programs  had  higher  costs. 


3.3.2  Cost  Estimating  Procedure  -  Method  I 


Cost  estimating  procedures  for  small  space  systems  are  covered 
exclusively  by  Method  I.  The  use  of  Method  I  is  also  appropriate 
for  major  acquisition  programs  which  are  only  minimally  defined  but 
where  the  development  phase  (VALID,  PSED,  PROD)  is  specified. 


The  estimating  data  base  is  the  typical  or  average  reliability 
program  task  costs  developed  from  thirteen  missile  and  space 
programs.  Two  options  are  presented  (Methods  IA  and  IB).  The  first 
option  is  appropriate  when  only  minimal  information  on  the  program 
engineering  budget  is  available.  The  second  option  assumes 
availability  of  an  estimate  of  the  engineering  budget  either  as  a 
given  or  as  estimated  from  Table  3.6. 


Both  options  utilize  typical  reliability  task  costs  measured  by 
percentage  of  engineering  budget  and  apply  these  against  engineering 
budget  estivates  to  produce  MK/year  statistics. 

3. 3. 2.1  Method  IA  -  Engineering  Budget  Unknown 

When  no  estimate  of  program  engineering  budget  is  available, 
reliability  program  and  task  costs  can  be  read  directly  from  Table 
3-7  .  These  data  are  the  typical  (average)  task  costs  observed  in 
the  modified  data  base.  Their  distribution  among  the  total 
reliability  program  is  shown  in  Table  3-9. 

3. 3. 2. 2  Method  IB  -  Engineering  Budget  Known  or  Estimated 

Method  IB  uses  an  engineering  budget  estimate  from  known  program 
data  or  from  Table  3-6  to  develop  reliability  programs  and  task 
MM/year  data.  Basic  to  Method  IB  is  the  typical  reliability  program 
task  costs  measured  as  a  percentage  of  engineering  budget  (Table 


TABLE  3-7 


AVERAGE  RELIABILITY  PROGRAM  AND  TASK 
EXPENDITURES  -  MM  PER  YEAR 


! 

J 

i 

4 

i 


. . :  ■ - 1 

TASK 

VALID 

MAJUK  rKUfjKJi 

FSED 

n 

PROD 

SMALL 

SPACE 

PROGRAM  PLAN 

.86 

.38 

.00 

.36 

MONITOR  &  CONTROL 

1.32 

13.71 

9.25 

.60 

SUBCONTRACTOR 

PROGRAM  REVIEWS 

1.29 

6.61 

1.92 

1.61 

FRACAS  &  FRB 

4.42 

40.3 

27.5 

.89 

ANALYSIS 

4.50 

19.07 

16.51 

.60 

FMECA 

1.30 

32.49 

— 

.10 

SCA 

— 

— 

— 

— 

ELEC.  PARTS/CIRC. 

1.6 

12.05 

— 

TOLERANCE  ANALYSIS 

PARTS 

2.34 

196.77 

53.33 

.75 

CRITICAL  ITEMS 

.60 

.50 

.19 

— 

EFFECTS  OF  TEST., 

1.62 

2.82 

3.12 

— 

STORAGE,  ETC. 

TESTING 

2.72 

37.68 

15.28 

.67 

ENG.  SUPPORT 

4.34 

23.21 

28.35 

1.33 

TOTAL 

26.91 

385.59 

135.45 

6.91 

TABLE  3-8 


AVERAGE  RELIABILITY  PROGRAM  AND  TASK 
EXPENDITURES  -  X  OF  ENGINEERING  BUDGET 


',,^*>»^PROGRAM/PHASE 

1  MAJOR  PROGRAM 

SMALL 

TASK 

VALID 

FSED 

PROD 

SPACE 

PROGRAM  PLAN 

.096 

.005 

— 

.09 

MONITOR  &  CONTROL 
SUBCONTRACTOR 

.147 

.174 

.280 

.16 

PROGRAM  REVIEWS 

.147 

.082 

.056 

.42 

FRACAS  &  FRB 

.492 

.5061 

.827 

.23 

ANALYSIS 

.501 

.236 

.495 

.16 

FMECA 

.144 

.405 

— 

.03 

SCA 

— 

— 

— 

— 

ELEC.  PARTS /CIRC. 
TOLERANCE  ANALYSIS 

.177 

.149 

— 

— 

PARTS 

.261 

2.458 

1.602 

.2 

CRITICAL  ITEMS 

.066 

.005 

.047 

— 

EFFECTS  OF  TEST., 
STORAGE,  ETC. 

.18 

.034 

.093 

— 

TESTING 

.303 

.472 

.458 

.18 

ENG.  SUPPORT 

.483 

.289 

.850 

.35 

TOTAL 

mm 

4.82 

4.67 

1.81 

TABLE  3-9 


METHOD  I  RELIABILITY  TASK  EXPENDITURES  - 
Z  OF  RELIABILITY  PROGRAM  BUDGET 


i 

> 


^*'"«N**^PROGRAM/PHASE 

MAJOR  PROGRAM 

SMALL 

SPACE 

TASK 

VALID 

FSED 

PROD 

PROGRAM  PLAN 

3.2 

.1 

— 

5.2 

MONITOR  &  CONTROL 
SUBCONTRACTOR 

4.9 

3.6 

6.0 

8.7 

PROGRAM  REVIEWS 

4.9 

1.7 

1.2 

23.3 

FRACAS  &  FRB 

16.4 

10.5 

17.7 

12.9 

ANALYSIS 

16.7 

4.9 

10.6 

8.7 

FMECA 

4.8 

8.4 

— 

1.4 

SCA 

— 

— 

— 

— 

ELEC.  PARTS /CIRC. 
TOLERANCE  ANALYSIS 

5.9 

3.1 

— 

— 

PARTS 

8.7 

51.0 

34.30 

10.9 

CRITICAL  ITEMS 

2.2 

.1 

.1 

— 

EFFECTS  OF  TEST., 
STORAGE,  ETC. 

6.0 

.7 

2.0 

— 

TESTING 

10.1 

9.8 

9.8 

9.7 

ENG.  SUPPORT 

16.1 

6.0 

18.2 

19.2 

TOTAL 

100.0 

100.0 

100.0 

100.0 
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The  procedure  described  below  is  summarized  in  a  companion 
worksheet,  Table  3-10.  An  example  calculation  is  shown  in  Table 
3-11. 


Step  1  -  Confirm  applicability  of  data  to  candidate 

system  and  program.  Reference  Section  3.1 

Step  2  -  Identify  system  type  (major  acquistion  or 

small  space)  and  development  phase  (VALID, 
FSED,  PROD)  if  major  acquisition.  For  more 
than  one  phase,  repeat  following  steps  for 
each  phase. 

Step  3  -  If  engineering  budget  is  unknown,  estimate 

budget  from  Table  3-6.  Reference  Section 
3.3.1. 

Step  4  -  Obtain  reliability  program  and  task  costs 

(percentage  of  engineering  budget)  from 
appropriate  column  of  Table  3-S. 

Step  5  -  Apply  percentage  figures  of  Step  4  to 

engineering  budget  figure  (Step  3) 

Step  6  -  Determine  Z  of  reliability  budget  values 

from  Table  3-9. 


TABLE  3-10  -  METHOD  1  WORKSHEET 

ENGINEERING  BUDGET  _ MM/YR  (ACTUAL) 


(LIST  ACTUAL  OR  ESTIMATED)  _ MM/YR  (ESTIMATED- TABLE  3-6) 


RELIABILITY  TASK 

X  ENG. 
BUDGET 

ANNUAL  REL. 
BUDGET-MM/YR 

X  REL. 
BUDGET 

(COLUMN  1) 

(COLUMN  2) 

(COLUMN  3) 

(COLUMN  4) 

PROGRAM  PLAN 

MONITOR  &  CONTROL 
SUBCONTRACTORS 

SUB 

PROGRAM  REVIEWS 

(ENTRIES 

FROM 

TABLE  3-8) 

(COL.  2X 
ESTIMATE 

OR  ACTUAL 
FROM  ABOVE) 

(ENTRIES 

FROM 

TABLE  3-9) 

FRACAS  &  FRB 

ANALYSIS 

FMECA 

SNEAK  CIRC.  ANAL. 

(SEE  DISCUSS 

ION  IN  SECTION 

4.3) 

ELEC.  PARTS/CIRC. 
TOLERANCE  ANALYSIS 

PARTS 

CRITICAL  ITEMS 

c 

EFFECTS  OF  TEST, 

STORAGE,  ETC. 

TESTING 

ENG.  SUPPORT 

TOTAL 

100Z 
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TABLE  3-11  -  EXAMPLE  METHOD  I  CALCULATION 

(MAJOR  ACQUISITION-VALID  PHASE) 

ENGINEERING  BUDGET  1 100  MM/YR  (ACTUAL) 

(LIST  ACTUAL  OR  ESTIMATED)  _ MM/YR  (ESTIMATED-TABLE  3-6) 


RELIABILITY  TASK 

X  ENG. 

BUDGET 

ANNUAL  REL. 
BUDGET-MM/YR 

%  REL. 
BUDGET 

(COLUMN  1) 

(COLUMN  2) 

(COLUMN  3) 

(COLUMN  4) 

PROGRAM  PLAN 

.096 

1.06 

3.2 

MONITOR  &  CONTROL 
SUBCONTRACTORS 

.147 

1.62 

4.9 

PROGRAM  REVIEWS 

.  147 

1.62 

4.9 

FRACAS  &  FRB 

.492 

5.41 

16.4 

ANALYSIS 

.501 

5.51 

16.7 

FMECA 

.1441 

1.58 

4.8 

SNEAK  CIRC.  ANAL. 

— 

— 

— 

ELEC.  PARTS/ CIRC. 
TOLERANCE  ANALYSIS 

.177 

1.95 

5.9 

PARTS 

.261 

2.87 

8.7 

CRITICAL  ITEMS 

.066 

.73 

2.2 

EFFECTS  OF  TEST, 

STORAGE,  ETC. 

.180 

1.98 

6.0 

TESTING 

.303 

3.33 

10.1 

ENG.  SUPPORT 

.483 

5.32 

16.1 

TOTAL 

3.00 

33.0 

100% 

3.3.3 


Cost  Estimating  Procedure  -  Method  II 


Cost  estimating  Method  II  is  limited  to  major  acquisition  missile 
programs  whose  characteristics  such  as  complexity,  configuration, 
hardware  status  are  known  or  can  be  accurately  estimated.  As  noted 
previously,  small  space  programs  are  covered  exclusively  by  Method 
I,  Section  3.3.2. 

Method  II  employs  a  set  of  linear  regression  equations  to  estimate 
reliability  task  costs  measured  as  a  percentage  of  engineering 
budget.  These  equations  relate  various  MIL-STD-785B  reliability 
task  costs  to  specific  system  characteristics.  A  set  of  ten  system 
characteristics  are  used  in  the  cost  estimating  equations.  This 
set,  along  with  a  numerical  scoring  criteria  for  use  in  the 
regression  equations,  is  shown  in  Table  3-12.  Most  of  the 
characteristics  are  scored  nominal  (0),  above  nominal  (positive 
score)  or  below  nominal  (negative  score).  Exceptions  to  this 
scoring  include  program  phase,  the  phase  II  term  to  account  for 
nonlinearities  and  the  readiness  requirement  scored  as  (1)  for 
present  or  (0)  for  not  present. 


53 


TABLE  3-12  -  PROGRAM/ SYSTEM  CHARACTERISTIC 

DEFINITIONS  &  SCORING  CRITERIA 


PROGRAM  PHASE 
(I) 

VALID 

FSED 

S  PROD 

-1 

0 

1 

VALID 

-1 

PROGRAM  PHASE 

FSED 

0 

(II) 

PROD 

1 

COST  EFFECTIVE 

-1 

BELOW  NOMINAL 

-.5 

RELIABILITY 

NOMINAL 

0 

CRITICALITY 

ABOVE  NOMINAL 

.5 

EXTREME 

1 

4999  OR  LESS 

-2 

5000  -  9999 

-1 

COMPLEXITY 

10000  -  19999 

0 

(PART  COURT) 

20000  -  49999 

1 

50000  -  OR  MORE 

2 

OFF  THE  SHELF 

-1 

HARDWARE  STATUS 

MODIFIED 

0 

NEW 

1 

999  OR  LESS 

-2 

1000  TO  1999 

-1 

HUMBER  OF 

2000  TO  3999 

0 

PART  TYPES 

4000  TO  7999 

1 

8000  OR  MORE 

2 

BELOW  SOA 

-1 

MISSION  REQ. 

NOMINAL 

0 

DIFFICULT 

1 

READINESS  REQ. 

YES  THEY  HAVE  ONE 

NO  THEY  DON’T 

1 

0 

PERCENTAGE  OF 

OZ  -  35Z 

-1 

FAILURE  RATE 

35Z  -  65Z 

0 

SUBCONTRACTED 

65Z  -  100Z 

1 

SINGLE  THREAD 

•1 

CONFIGURATION 

. 

0 

RESTORABLE 

1 

i 


5 


w 

k 


The  regression  equations  require  a  definitive  (inputted)  score  for 
the  phase  I  and  phase  II  terms.  All  other  characteristic  scores  can 
default  to  zero  (nominal)  if  no  definition  of  the  particular 
characteristic  is  available. 

Equations  were  developed  for  nine  of  the  thirteen  reliability  task 
cost  categories.  The  remaining  task  costs  are  estimated  by  means  of 
the  typical  costs  developed  for  Method  I. 

For  each  task,  three  cost  estimating  regression  equations  using 
combinations  of  one,  two,  and  three  system  characteristics  were 
developed.  Supplementary  investigations  had  shown  insignificant 
impact  on  task  costs  beyond  consideration  of  combinations  of  up  to 
three  characteristics.  For  reliability  program  cost  estimating 
purposes,  one  of  the  three  candidate  equations  was  selected  for  each 
task.  This  set,  labeled  "Best"  is  shown  in  Table  3-13.  Equation 
selection  was  based  on  an  evaluation  of  changes  in  the  standard 
error  estimate  and  regression  coefficients  resulting  from  each 
characteristic  addition.  Further  details  of  this  process  along  with 
the  complete  set  of  regression  equations  are  contained  in  Appendix  B. 

The  most  consistent  attribute  in  the  set  of  cost  estimating 
equations  relates  to  program  phase;  either  Phase,  Phase  II  or  both. 
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The  dominance  of  program  phase  is  further  illustrated  by  comparing 
the  respective  program  attribute  coefficients  in  Table  3-13.  In 
general,  those  coefficients  associated  with  phase  parameters  are 
significantly  higher  than  those  associated  with  other  program 
characteristics  or  attributes.  Generally,  the  task  cost  equations 
can  be  viewed  as  an  average  value.  A,  which  is  adjusted  by  program 
phase,  and  to  a  lesser  extent,  by  other  system  attributes.  With  few 
exceptions,  e.g.,  the  readiness  attribute  on  storage  studies,  the 
influence  of  these  lesser  attributes  is  neither  intuitive  nor 
consistent.  While  the  exact  reasons  for  these  results  are  not 
obvious,  a  masking  of  relationships  between  the  overall  program  cost 
data,  and,  by  association,  the  reliability  cost  data,  and  system 
attributes  is  a  factor.  This  means  that  for  more  complex  and  state 
of  the  art  programs,  not  only  are  absolute  reliability  task  costs 
increased,  but  also  overall  program  costs  are  increased 
proportionally.  This,  in  effect,  disguises  any  influence  on  the 
chosen  measure.  Measures  other  than  percentage  of  engineering 
budget  (e.g.,  MM/YR)  could  have  been  employed  to  avoid  the  masking, 
however,  these  were  not  used  in  the  correlation  analysis  to  preclude 
domination  by  results  from  a  few  large  programs.  Despite  these 
inconsistencies,  the  tabled  equations  and  correlations  are, 
nevertheless,  accurate  reflections  of  the  respective  program 
attributes  on  task  costs  as  observed  from  the  modified  data  base. 
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As  such,  they  can  be  used  to  predict  reliability  task  costs  for 
systems  similar  to  those  described  herein. 


PROCEDURE : 


Step  1  - 


Step  2  - 

Step  3  - 

Step  4  - 


Step  5  - 


Step  6  - 

Step  7  - 


Confirm  applicability  of  data  to  candidate  system  and 
program.  Reference  Section  3.1 

Define  required  system  characteristics  and  score  per 
Table  3-12  criteria. 

If  engineering  budget  is  unavailable,  estimate  budget 
from  Table  3-6.  Reference  Section  3.3.1 
Exercise  regression  equations.  Table  3-13  and 
Table  3-15,  using  system  characteristic  scores  from 
Step  2. 

Obtain  cost  percentages  for  tasks  not  covered  by 
regression  equations  from  Table  3-8. 

Apply  percentage  figure  of  Steps  4  and  5  to 
engineering  budget  figure  of  Step  3. 

Determine  Z  of  reliability  budget  values  by  taking 
ratios  of  task  costs  to  total  reliability  program 
cost. 
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TABLE  3-14 


METHOD  II  SUMMARY  WORKSHEET 


ENGINEERING  BUDGET: 

(LIST  ACTUAL  OR  ESTIMATED) 


MM/YR  (ACTUAL) 

MM/YR  (ESTIMATED-TABLE  3-6) 


Z  ENG. 

ANNUAL  REL. 

Z  REL. 

RELIABILITY  TASK 

BUDGET 

BUDGET-MM/YR 

BUDGET 

(COLUMN  1) 

(COLUMN  2) 

(COLUMN  3) 

(COLUMN  4) 

PROGRAM  PLAN 

(ENTRY  FROM 

(COL.  2  X 

(ENTRIES  - 

TABLE  3-8) 

ESTIMATE 

COL.  2 

MONITOR  &  CONTROL 
SUBCONTRACTORS 

PROGRAM  REVIEWS 

FRACAS  &  FRB 

ANALYSIS 

FMECA 

SNEAK  CIRC.  ANAL. 

ELEC.  PARTS/CIRC. 
TOLERANCE  ANALYSIS 

PARTS 

CRITICAL  ITEMS 

EFFECTS  OF  TEST, 
STORAGE,  ETC. 

TESTING 

ENG.  SUPPORT 


TOTAL 


(ENTRIES 
FROM  TABLE 
3-15  WORK¬ 
SHEET) 


OR  ACTUAL 
FROM  ABOVE) 


ENTRIES/ 
COL.  2 
TOTAL) 


(SEE  DISCUSSI3N  IN  SECTION  4.3 

(ENTRY  FROM 
TABLE  3-8) 


(ENTRIES 
FROM  TABLE 
3-15  WORK¬ 
SHEET) 


(ENTRY  FROM 
TABLE  3-8) 


TABLE  3-15  -  RELIABILITY  TASK  COST  REGRESSION 

EQUATION  WORKSHEET 


RELIABILITY  TASK 

CHARACTERISTIC 

SCORE 

MULT. 

1  COST  | 

SUB 

TOTAL 

MONITOR  &  CONTROL 

CONSTANT 

.22 

.22 

SUBCONTRACTORS 

PHASE  (I) 

— 

— 

STATUS 

— 

— 

— 

PROGRAM  REVIEWS 

CONSTANT 

.08 

00 

© 

• 

COMPLEXITY 

— 

— 

FRACAS  &  FRB 

CONSTANT 

.64 

1 

.64 

PHASE  (I) 

— 

.22 

— 

PHASE  (II) 

— 

.17 

— 

X  F.R.  SUB. 

— 

-.21 

— 

— 

ANALYSIS 

CONSTANT 

.26 

1 

.26 

PHASE  (II) 

— 

.22 

— 

CRITICALITY 

— 

-.08 

— 

STATUS 

— 

-.06 

— 

— 

FMECA 

CONSTANT 

.63 

1 

.63 

PHASE  (II) 

— 

-.53 

— 

STATUS 

— 

-.27 

— 

— 

PARTS 

CONSTANT 

2.78 

1 

2.78 

PHASE  (I) 

— 

.68 

— 

PHASE  (II) 

— 

-1.90 

~ 

— 

CRITICAL  ITEMS 

PHASE  II 

— 

.05 

__ 

COMPLEXITY 

— 

.03 

NO.  PART  TYPES 

— 

-.04 

— 

— 

EFFECTS  OF  TESTING, 

READINESS 

.28 

.... 

STORAGE,  ETC. 

STATUS 

— 

.07 

— 

NO.  PART  TYPES 

— 

.04 

— 

— 

TESTING 

CONSTANT 

.37 

1 

.37 

CRITICALITY 

-.07 

— 

COMPLEXITY 

.03 

— 

STATUS 

" 

.03 
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TABLE  3-16  -  EXAMPLE  METHOD  II  SUMMARY  WORKSHEET 

(MAJOR  ACQUISITION  -  PROD  PHASE) 

ENGINEERING  BUDGET:  _ MM/YR  (ACTUAL) 

(LIST  ACTUAL  OR  ESTIMATED)  3500  MM/YR  (ESTIMATED-TABLE  3-6) 


RELIABILITY  TASK 

X  ENG. 
BUDGET 

ANNUAL  REL. 
BUDGET-MM/YR 

%  REL. 
BUDGET 

(COLUMN  1) 

(COLUMN  2) 

(COLUMN  3) 

(COLUMN  4) 

PROGRAM  PLAN 

0 

0 

0 

MONITOR  &  CONTROL 
SUBCONTRACTORS 

.23 

8.05 

4.98 

PROGRAM  REVIEWS 

.08 

2.80 

1.73 

FRACAS  &  FRB 

1.03 

36.05 

22.32 

ANALYSIS 

.38 

13.30 

8.23 

FMECA 

0 

0 

0 

SNEAK  CIRC.  ANAL. 

— 

— 

— 

ELEC.  PARTS/CIRC. 

TOLERANCE  ANALYSIS 

0 

0 

0 

PARTS 

1.56 

54.60 

33.80 

CRITICAL  ITEMS 

.01 

.35 

.22 

EFFECTS  OF  TEST, 

STORAGE,  ETC. 

.11 

3.85 

2.38 

TESTING 

.365 

12.78 

7.91 

ENG.  SUPPORT 

.85 

29.75 

18.42 

TOTAL 

4.615 

161.53 

100Z 

TABLE  3-17 


EXAMPLE  RELIABILITY  TASK  COST 
REGRESSION  EQUATION  WORKSHEET 


I  RELIABILITY  TASK 

CHARACTERISTIC 

SCORE 

MULT. 

i  COST 

SUB 

TOTAL 

MONITOR  &  CONTROL 

CONSTANT 

.22 

1 

.22 

SUBCONTRACTORS 

PHASE  (I) 

1 

.08 

.08 

STATUS 

1 

-.07 

-.07 

.23 

PROGRAM  REVIEWS 

CONSTANT 

.08 

1 

• 

o 

00 

COMPLEXITY 

0 

-.04 

.08 

FRACAS  &  FRB 

CONSTANT 

.64 

1 

.64 

PHASE  (I) 

1 

.22 

.22 

PHASE  (II) 

1 

.17 

.17 

%  F.R.  SUB. 

0 

-.21 

0 

1.03 

ANALYSIS 

CONSTANT 

.26 

1 

.26 

PHASE  (II) 

1 

.22 

.22 

CRITICALITY 

.5 

-.08 

-.04 

STATUS 

1 

-.06 

-.06 

• 

u> 

00 

FMECA 

CONSTANT 

.63 

1 

.63 

PHASE  (II) 

1 

-.53 

-.53 

STATUS 

1 

-.27 

-.27 

&> 

PARTS 

CONSTANT 

2.78 

1 

2.78 

PHASE  (I) 

1 

.68 

.68 

PHASE  (II) 

1 

-1.90 

-1.90 

1.56 

CRITICAL  ITEMS 

PHASE  II 

1 

.05 

.05 

COMPLEXITY 

0 

.03 

0 

NO.  PART  TYPES 

1 

t.04 

-.04 

.01 

EFFECTS  OF  TESTING, 

READINESS 

0 

.28 

0 

STORAGE,  ETC. 

STATUS 

1 

.07 

.07 

NO.  PART  TYPES 

1 

.04 

.04 

.11 

TESTING 

CONSTANT 

.37 

1 

.37 

CRITICALITY 

.5 

-.07 

-.035 

COMPLEXITY 

0 

.03 

0 

STATUS 

1 

.03 

.03 

.365 

&>  NEGATIVE  VALUE  -  INTERPRET  AS  ZERO  COST  FOR  THIS  TASK  DURING  PROD 
PERIOD 


4.0 


RELIABILITY  PROGRAM  TAILORING 


Apart  from  cost  considerations,  comprehensive  development  of 
reliability  program  and  task  tailoring  is  beyond  the  scope  of  this 
guide.  A  recommendation  is  that  a  follow-on  study  be  pursued  in  the 
area  of  reliability  tailoring  as  a  function  of  task  effectiveness. 
The  purpose  of  such  a  study  would  be  to  determine  the  various 
effectiveness  relationships  between  program  or  system 
characteristics  and  task  selection  and  implementation.  Special 
emphasis  would  be  placed  on  the  cost  driver  tasks,  tasks  which  are 
undergoing  changes  or  are  in  development,  and  tasks  which  interface 
with  other  disciplines. 

4.1  Time  Phasing  of  Tailoring 

Prior  to  the  concept  development  phase,  a  system  is  known  only  to 
the  extent  that  it  must  satisfy  certain  operational  needs.  The 
configuration,  complexity  and  state-of-the-art  developments  are  yet 
to  be  fully  determined.  However,  at  this  point,  the  reliability 
manager  or  monitor  is  often  called  upon  to  begin  the  specification 
process  for  the  reliability  program.  Two  observations  are  pertinent 
here,  and  are  listed  below: 
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1)  Th  reliability  task  tailoring  process  is  dynamic, 
continuing  from  concept  development  through 
validation,  full-scale  engineering  development 
and  production. 

2)  The  contractor(s)  should  be  fully  involved  in  the 
tailoring  process. 

These  two  observations  were  developed  from  responses  and  follow-up 
interviews  to  the  questionnaire  discussed  in  the  Overview  of  Section 
2.0.  A  partial  summary  of  the  responses  is  provided  in  Table  4-1. 

Answers  to  the  first  question  regarding  the  phased  timing  of 
tailoring  indicate  a  36%  positive  response  for  "tailoring"  during 
the  concept  development  or  contract  definition  phase.  Other 
responses  were  20%  for  tailoring  to  be  included  in  the  contractor(s) 
development  and  production  proposal  and  28%  in  favor  of  tailoring  as 
a  topic  of  negotiation  at  the  time  of  contract  award.  Follow-up 
interviews  brought  out  the  need  for  additional  considerations 
including  that  of  contractor  involvement  in  post  award  tailoring 


decisions 


TABLE  4-1 


QUESTIONNAIRE  RESPONSE  TO  TAILORING  DECISIONS 


DURING  WHAT  PHASE  SHOULD  THE  TAILORING 

PROCESS  OCCUR? 

H3SSBI 

NO. 

% 

a.  CONTRACT  DEFINITION  (STUDY  TASK) 

9 

36 

b.  FSED/PROD.  PROPOSAL  (CONTR.  RESPONSE) 

5 

20 

C.  CONTRACT  AWARD  (CUST. /CONTR.  NEGOT.) 

D 

28 

d.  OTHER 

B 

16 

25 

GIVEN  THAT  A  TAILORING  METHOD  HAS  BEEN  DEVELOPED, 

HOW  SHOULD  IT  BE  MODIFIED  IF  THERE  IS  A  SHORTAGE 

OF  CONTRACT  DOLLARS? 

RESPONSE  | 

NO. 

% 

a.  PRO-RATA  SCALING 

1 

B 

b.  DELETION  OF  SOME  TASKS  ON  PRIORITY  BASIS 

11 

B 

c.  CHANGES  IN  EXTENT  OR  COVERAGE  OF  TASKS 

9 

1 

d.  CHANGES  IN  DEPTH  OF  APPL.  OF  TASKS 

5 

B 

e.  OTHER 

1 

B 

27 

WHAT  IN  YOUR  OPINION  IS  THE  MOST  SIGNIFICANT 

STUMBLING  BLOCK  TO  IMPLEMENTING  AN  ACCEPTABLE 
RELIABILITY  PROGRAM? 

RESPONSE  | 

NO. 

% 

a.  PROGRAM  FUNDS 

B 

37 

b.  CONTR. /CUST.  MANAG.  BIAS  OR  APATHY 

B 

37 

C.  IMPROPER  TAILORING 

B 

16 

d.  OTHER 

B 

10 

19 
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The  general  lesson  to  be  learned  is  that  tailoring  is  not  a 
"one-time"  static  process  developed  by  the  government  reliability 
manager  or  monitor  and  impj^ed  on  a  contractor,  but  a  process  which 
is  dependent  on  information  developed  over  time  and  which  is 
dependent  on  a  contractor's  unique  knowledge  of  the  program  and  the 
system.  Two  examples  will  illustrate  this  point.  On  one  program 
the  requirement  for  a  reliability  oriented  Failure  Mode,  Effects  and 
Criticality  Analysis  (FMECA)  was  deleted  because  the  results  would 
have  been,  at  best,  marginal  for  this  particular  program.  This 
decision  was  based  on  the  configuration  of  the  system,  dominantly 
"single  thread",  which  was  not  fully  apparent  at  the  time  of 
specifying  the  FMECA  task.  Another  example  involves  a  program  where 
a  reliability  "improvement"  task  was  funded  and  implemented  during 
the  production  phase  and  after  the  system  had  been  introduced  into 
operational  service.  The  effectiveness  of  this  particular  task  was 
based  on  the  discovery  and  correction  of  deficiencies  which  could  be 
exposed  only  as  a  result  of  operation  in  the  military  environment 
including  that  of  the  prevailing  maintenance  practices. 

In  both  of  the  above  examples,  task  cost  effectiveness  was  the  basic 
issue.  In  the  one  case,  the  task  pay-off  was  less  than  the  task 
cost  and  the  savings  were  transferred  to  another  task.  In  the  other 
case,  the  additional  expenditure  on  reliability  improvement  did 


pay-off,  resulting  in  a  significant  increase  in  system  reliability. 
The  latter  case  is  an  argument  for  advance  commitment  of  funds  to 
this  type  of  product  improvement  effort. 

The  second  question  dealt  with  the  problem  of  modifying  reliability 
program  efforts  which  had  been  well  tailored  to  a  particular  program 
but  which  could  not  be  supported  with  allocated  contract  funds. 

Here  the  problem  was  how  to  achieve  the  most  effective  program  at  a 
reduced  cost.  There  was  virtually  no  positive  response  to  the  idea 
of  a  scaling  down  of  all  tasks  in  proportion  to  the  budget 
reduction.  The  most  effective  measures  were  thought  to  be  either  a 
deletion  of  some  tasks  on  an  effectiveness  priority  basis  or  a 
limitation  on  the  coverage  of  tasks  based  on  equipment  criticality. 

A  final  question  concerned  the  respondents'  opinions  on  the  most 
significant  stumbling  block  to  implementing  an  acceptable 
reliability  progam.  While  only  10Z  felt  that  improper  tailoring  was 
at  fault,  an  equally  divided  response  of  37Z  each  felt  the  fault  was 
with  program  funding  limitations  or  bias  or  apathy  on  the  part  of 
the  contractor  or  customer  management. 


4.2 


Tailoring  Mechanics 


Development  of  an  effective  reliability  program  implies  a  tailoring 
of  tasks  which  recognizes  the  unique  features  of  a  particular 
program  or  system.  The  tailoring  process  can  involve  more  than  a 
selection  of  tasks  from  MIL-STD-785  or  -1543.  It  can  involve 
development  of  new  tasks  or  significant  shaping  in  depth  or  extent 
of  application  of  selected  tasks.  A  prerequisite  of  effective 
tailoring,  apart  from  the  knowledge  of  program/task  costs,  is 
knowledge  of  the  program/ system  and  its  various  features  which 
impact  reliability. 

For  Air  Force  missile  and  space  systems,  reliability  program 
tailoring  basically  involves  the  modification  of  the  task  provisions 
of  MIL-STD-785  or  MIL-STD-1543 .  These  modifications  are  made  to 
suit  the  nature  and  circumstances  of  a  particular  program/ system. 
There  are  three  primary  modes  of  modification  or  tailoring: 

•  Selection  of  only  those  tasks  which  have  "pay-off" 
for  a  given  program/system  (Select  Category). 

•  Limiting  application  of  tasks  to  those  system 
components  where  there  is  "pay-off"  (Extent  Category). 

•  Controlling  the  depth  of  application  to  those  levels 
where  these  is  "pay-off”  (Depth  Category). 


Additionally,  tasks  could  be  implemented  or  reported  in  some  special 
manner,  or  a  task  not  listed  in  -785  or  -1543  could  be  added  to  suit 
the  needs  of  a  particular  program.  In  the  case  of  the  former, 
options  are  often  available  on  the  particular  way  a  task  is 
implemented;  e.g.,  the  selection  of  a  test  plan  from  MIL-STD-781  or 
selection  of  a  particular  derating  guide.  For  the  latter  case,  it 
is  sometimes  advantageous  to  add  tasks  when  it  becomes  apparent  that 
by  doing  so  a  significant  gain  in  reliability  can  be  achieved;  e.g., 
the  addition  of  a  reliability  improvement  task  after  the  system's 
introduction  into  operational  service. 

Examples  of  the  tailoring  categories  described  above  are  as  follows: 

SELECT  CATEGORY  -  A  reliability  oriented  FMECA  task 
is  omitted  because  the  system  is  a  single  thread 
(series)  configuration  at  all  levels  and  the  effect 
of  any  failure  is  always  loss  of  mission.  This  does 
not  preclude  peiformance  of  this  type  of  analysis 
for  maintainability  or  other  non-reliability  reasons. 

EXTENT  CATEGORY  -  A  system  component  has  been 
previously  qualified  for  the  same  application  and 
environment  allowing  this  component  to  be  exempt 
from  certain  task  applications. 
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DEPTH  CATEGORY  -  A  system  is  in  the  Concept  Development 
(C/D)  Phase  with  definition  lacking  below  the  functional 
component  level.  Depth  of  analysis  for  this  phase 
could  be  limited  to  "similar  system"  or  MIL-HDBK-217 
"Parts  count  Reliability  Prediction"  (Section  5.2) 
methods . 

Realizing  the  ways  reliability  programs  or  tasks  can  be  modified  is 
the  first  step  in  the  tailoring  process.  The  second  step  is  to 
determine  the  criteria  for  application  of  these  various  types  of 
modifications.  Here  the  concern  is  to  determine  what  features  or 
characteristics  of  a  program  or  system  have  significant  bearing  on 
the  choice  and  structuring  of  reliability  tasks.  These 
characteristics  are  basically  the  same  as  were  investigated  for  cost 
except  that  the  focus  is  changed.  The  cost  investigation  focused. on 
how  program/task  costs  vary  as  a  function  of  these  characteristics. 
The  tailoring  problem  considers  how  the  basic  reliability 
program/task  are  modified  as  a  function  of  these  same 
characteristics.  Together,  the  two  investigations  allow  development 
of  a  reliability  program  which  is  both  "tailored"  to  a  given 
program/ sys tern  and  one  which  is  sensitive  to  cost  constraints;  i.e.. 


cost  effective 


The  basic  characteristics  discussed  in  the  previous  sections  were 
developed  by  considering  what  type  of  reliability  program 
modifications  were  likely  to  have  an  impact  on  program/task  costs. 
Sources  of  these  modifications  stemmed  from  consideration  of  the 
following  aspects  of  a  system  and  its  procurement  program: 

•  SYSTEM  EMPLOYMENT/DEPLOYMENT  CHARACTERISTICS 
(Criticality  of  reliability  to  achievement  of 
objectives,  storage  and  service  life  require¬ 
ments,  maintenance  concepts,  etc.) 

•  MISSION  CHARACTERISTICS 

(Mission  duration,  environments,  duty  cycles, 
numerical  reliability  requirement,  etc.) 

•  SYSTEM  CHARACTERISTICS 

(Hardware /software  development  status, 
complexity,  configuration,  etc.) 

•  PROGRAM  CHARACTERISTICS 

(Schedule,  unit  costs,  contractor  role, 
phase,  etc.) 
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Setting  aside  the  cost  considerations,  the  tailoring  process 
concentrates  on  shaping  a  reliability  program  as  influenced  by  these 
characteristics.  Examples  of  these  influences  and  their 
significance  can  be  seen  in  the  following  discussions  on  program 
phase,  system  configuration  and  system  deployment  mode. 

Table  4-2,  taken  from  MIL-STD-7 85B ,  provides  initial  tailoring 
criteria  in  terms  of  program  phase.  The  applicability  of  each 
reliability  task  is  noted  for  each  of  four  program  phases.  It 
further  defines  those  special  cases  requiring  considerable 
interpretation  or  reference  to  other  military  standards.  These 
initial  criteria  do  not,  however,  address  the  impact  of  other 
program/ sy stem  characteristics  on  the  tailoring  process. 

Program  phase  is  a  significant,  if  not  the  most  significant,  program 
characteristic.  Phase  impacts  not  only  task  selection,  but  also  is 
a  large  factor  in  the  required  depth  and  extent  of  any  selected 
task.  For  example,  the  FRACAS/FRB  task  is  a  considerably  greater 
expenditure  during  the  PROD  phase  than  during  either  the  VALID  or 
FSED  phases.  Similarly,  the  FKECA  task  is  approximately  8-10%  of 
the  reliability  budget  during  FSED  while  falling  to  zero  during  the 
PROD  phase.  These  large  expenditure  differences  reflect  the  more 
basic  differences  in  task  application  or  modification  as  uncovered 
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TABLE  4-2 


TASK  APPLICATION  MATRIX 


NO. 

TASK  TITLE 

TYPE 

PHASE  ! 

CONC. 

VALID 

FSED 

PROD 

101 

Reliability  Program  Plan 

MGT 

S 

S 

G 

G 

102 

Monitor/Control  Sub.  &  Suppl. 

MGT 

S 

S 

G 

G 

103 

Program  Reviews 

MGT 

s 

S(2) 

G(2) 

G(2) 

104 

Failure  Reporting  Etc  (FRACAS) 

ENG 

NA 

S 

G 

G 

105 

Failure  Review  Board 

MGT 

NA 

S(2) 

G 

G 

201 

Reliability  Modeling 

ENG 

s 

S(2) 

G(2) 

GC(2) 

202 

Reliability  Allocations 

ACC 

s 

G 

G 

GC 

203 

Reliability  Predictions 

ACC 

S 

S(2) 

G(2) 

GC(2) 

204 

Failure  Modes,  ETC  (FMECA) 

ENG 

S 

S(l,2) 

0(1,2) 

GC(1,2) 

205 

Sneak  Circuit  Analysis  (SCA) 

ENG 

NA 

NA 

G(  1) 

GC(1) 

206 

Elec  Parts/Circ  Tol  Anal 

ENG 

NA 

NA 

G 

GC 

207 

Parts  Program 

ENG 

S 

S(2,3) 

G(2) 

G(2) 

208 

Reliability  Critical  Items 

MGT 

S(l) 

S(l) 

G 

G 

209 

Effects  of  Funct  Test  Etc. 

ENG 

NA 

S(l) 

G 

GC 

301 

Env.  Stress  Screen  (ESS) 

ENG 

NA 

S 

G 

G 

302 

Rel.  Develop/Gwth  Test 

ENG 

NA 

S(2) 

G(2) 

NA 

303 

Rel.  Qual  Test 

ACC 

NA 

S(2) 

G(2) 

G(2) 

304 

Prod  Rel  Accpt  Test 

ACC 

NA 

NA 

S 

G(2 , 3) 

ACC 

ENG  -  Reliability  Engineering 
MGT  -  Management 
S  -  Selectively  Applicable 
G  -  Generally  Applicable 
GC  -  Generally  Applicable  to 
to  Design  Changes  Only 


NA  -  Not  Applicable 

(1)  -  Requires  Considerable 

Interpretation  of  Intent 
to  be  Cost  Effective 

(2)  -  MIL-STD-785  Not  Primary 

Implem.  Rqmt.  Reference 
SOW  or  Other  MIL-STDS 
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by  consideration  of  the  phase  characteristics.  The  dynamics  of  the 
acquisition  process,  including  the  evolution  of  the  design  from 
concept  to  detail  to  hardware,  has  associated  with  it  a  "data 
availability"  factor  which  greatly  affects  the  depth  of  analysis 
oriented  reliability  tasks. 

System  configuration  as  it  refers  to  whether  a  system  is  "single 
thread”  (series),  redundant  or  restorable,  is  another  characteristic 
which  can  be  significant  to  tailoring.  The  previous  example,  citing 
deletion  of  the  reliability  oriented  FHECA  for  a  single  thread 
configuration,  is  but  one  case.  Several  of  the  other  reliability 
tasks  can  also  be  affected.  With  redundant  and  restorable 
configurations,  analyses  tasks  become  more  complex  and  must 
interface  with  other  analyses  of  failure  detection,  maintainability, 
etc.  Additionally,  decisions  on  parts  quality,  derating,  and 
testing  can  be  affected  by  this  characteristic.  More  generally, 
redundant  or  restorable  configurations  represent  an  exercise  of  a 
fundamental  reliability  improvement  option  with  implications  on  the 
effectivity  of  other  improvement  options  such  as  part  quality  or 
reliability  growth  testing. 


As  a  final  example,  characteristics  steaming  from  a  system's 
deployment  mode  are  considered.  Many  systems  spend  most  of  their 


f. 
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lifetimes  in  a  so  called  "storage"  mode.  A  requirement  of  such 
systems  is  to  have  a  high  readiness  reliability;  i.e.,  the  system 
should  have  a  high  probability  of  being  in  an  operable  state  when 
called  upon  for  use.  The  implications  of  this  requirement  affect 
several  reliability  tasks.  First,  in  the  VALID  phase  there  is  an 
increased  emphasis  on  analysis  and  development  of  data.  Also  during 
this  phase,  special  test  programs  might  be  initiated  to  investigate 
long  term  reliability  properties  of  parts  and  components.  During 
all  phases,  interface  analyses  involving  maintainability  and 
logistics  disciplines  must  be  accomplished  to  assure  compatibility 
with  maintenance  and  logistics  concepts.  In  general,  as  the 
reliability  technology  develops  to  more  fully  address  this 
particular  deployment  mode,  task  modifications  and  additions  can  be 
expected. 

The  above  examples  addressed  some  tailoring  implications  stemming 
from  consideration  of  three  different  program/ system 
characteristics.  There  are  many  other  considerations;  however, 
these  can  be  addressed  in  a  similar  manner  by  examining  the 
implications  of  the  characteristic  on  the  reliability  program  and 


constituent  tasks 


4.3 


Tailoring  Notes 


The  preceeding  paragraphs  discussed  the  general  procedures  and 
considerations  necessary  for  tailoring  of  KIL-STD-785B  reliability 
tasks  to  develop  a  cost-effective  program.  To  supplement  the 
general  tailoring  discussion,  specific  notes  regarding  task 
tailoring  and  system  characteristic  influences  on  task  costs  are 
presented  in  this  section.  These  notes  are  a  set  of  observations 
and  "rules  of  thumb"  accumulated  during  the  data  collection, 
screening,  and  analysis  process. 

As  noted  in  Section  3.3,  the  cost  statistics  collected  contained 
entries  reflecting  either  explicit  or  implicit  tailoring  of 
reliability  tasks.  A  careful  screening  of  the  data  was  necessary  to 
develop  a  homogenous  data  set  for  analysis  purposes.  This  screening 
process,  together  with  the  regression  analyses,  provided  valuable 
insight  into  the  tailoring  process  and  its  impact  on  reliability 
program  costs. 

This  collection  of  notes  is  not  intended  to  be  a  detailed  treatment 
of  reliability  task  cost  tailoring.  It  is  presented  only  to  provide 
insight  into  the  process  and  to  document  some  historical  reliability 
task  cost  tailoring  decisions. 
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The  data  is  structured  by  task;  first  displaying  the  task  cost  as  a 
percentage  of  the  reliability  budget  for  each  of  the  program 
categories  (taken  directly  from  the  summary  of  Table  3-9),  and 
followed  with  comments  on  the  cost  impact  of  any  tailoring  decision. 


RELIABILITY  PROGRAM  PLAN 

VALID  FSED  PROD  SMALL  SPACE 

3.2%  .1%  —  5.2% 

The  costs  associated  with  the  development  of  a  reliability  program 
plan  are  relatively  small.  Because  the  reliability  program  plan  is 
a  contractual  document  defining  the  remaining  reliability  effort, 
little  tailoring  is  required  for  this  task.  It  should  also  be  noted 
that  reliability  program  plans  written  to  M1L-STD-785B  appear  (based 
on  one  sample)  to  be  more  costly  than  those  written  to  MIL-STD-785A 
due  to  the  additional  detail  required. 

SUBCONTRACTOR  CONTROL 

VALID  FSED  PROD  SMALL  SPACE 

4.9%  3.6%  6.0%  8.7% 
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Subcontractor  control  is  one  of  the  nominal  cost  reliability  tasks 
consuming  some  4-6%  of  the  total  reliability  budget.  The  primary 
functions  of  this  task  are  to  assure  that  the  reliability 
requirements  imposed  upon  subcontractors  are  supportive  of  the 
system  requirements  and  to  assure  subcontactor  performance  and 
progress.  Because  of  the  basic  nature  of  this  task  and  its  nominal 
costs,  little  historical  tailoring  activity  was  observed.  It  is  not 
foreseen  as  a  significant  tailoring  candidate  for  system  or 
associate  level  contracts. 

PROGRAM  REVIEWS 

VALID  FSED  PROD  SMALL  SPACE 

4.9%  1.7%  1.2%  23.3% 

For  major  acquisition  programs,  program  review  is  another  of  the 
nominal  reliability  program  tasks  comprising  less  than  5%  of  the 
reliability  effort.  As  a  result  cost  tailoring  of  this  task  is  not 
considered  significant. 

In  the  raw  data  base,  one  entry  relating  to  program  reviews  showed 
the  effects  of  a  non-typical  application.  A  PROD  program  had  large 
program  review  costs.  This  program  included  a  major  "technology 


1 

I 


.1 


i 

i 

> 


.V 


transfer"  of  a  European  missile  design  to  U.S.  production  with  an 
attendant  increase  in  review  and  coordination  efforts  (approximately 
10%  of  the  total  reliability  effort). 


•  ! 


For  small  space  programs,  program  reviews  were  the  major  reliability 
program  cost  category.  These  programs  can  be  typified  as  technology 
integration  efforts  as  contrasted  to  technology  development  efforts. 
With  this  situation,  high  program  review  costs  are  not  unexpected. 


FRACAS  &  FRB 

VALID 

FSED  PROD 

SMALL  SPACE 

16.4% 

10.5%  17.7% 

12.9% 

The  FRACAS  and  FRB  activities  were  found  to  be  major  and  consistent 
cost  drivers  for  all  program  types  and  phases  and,  as  such,  present 
significant  tailoring  opportunity. 

The  historical  task  costs  were  found  to  be  sensitive  to  actual 
hardware  development  and  testing.  One  VALID  program  had  minimal 
FRACAS  and  FRB  costs  due  to  lack  of  a  test  program  in  the  basic 
effort.  Thus,  for  tailoring  and  estimating  of  FRACAS  and  FRB  costs, 
consideration  must  be  given  to  the  scope  of  test  program. 
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The  regression  analysis  found  a  direct  correlation  between  FRACAS 
and  FRB  costs  and  the  percentage  of  the  failure  rate  subcontracted 
i.e.,  the  higher  the  percentage  subcontracted  out,  the  less  the 
FRACAS  &  FRB  costs.  This  result  was  expected  due  to  decreased 
contractor  testing  activity  . 

A  final  observation  on  tailoring  FRACAS  &  FRB  activities  is  that  of 
cross-coupling  with  other  reliability  efforts.  As  noted  above,  a 
reduced  test  program  results  in  reduced  FRACAS  and  FRB  costs.  Also 
these  two  tasks  categories,  FRACAS/FRB  and  testing,  can  affect  the 
analysis  task.  An  example  is  a  VALID  effort  designed  to  evaluate 
the  trade,  "demonstration  flight  tests  vs  expanded  analyses, "as  it 
relates  to  developing  sufficient  confidence  in  the  missile  design  to 
proceed  to  FSED. 


RELIABILITY  ANALYSES 


VALID 

FSED 

PROD 

SHALL  SPACE 

16.72 

4.9% 

10.6% 

8.7% 

Reliability  analysis  is  one  of  the  basic  and  cost  consistent 
reliability  tasks.  The  phase  dependency  demonstrated  in  the  above 
table  is  confirmed  by  the  correlation  analyses.  No  tailoring 
activit  ,  other  than  that  accorded  to  phase,  was  observed  in  the 
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collected  cost  statistics.  One  opportunity  for  tailoring  includes 
the  trade  of  analyses  vs  test  activities  outlined  previously. 


FMECA 

VALID 

FSED 

PROD 

SMALL  SPACE 

4.8% 

8.4% 

— 

1.4% 

The  FMECA  task,  although  not  an  extreme  cost  driver,  was  one  of  the 
more  heavily  tailored  tasks.  As  can  be  noted  in  the  above 
percentage  display,  the  FMECA  task  is  phase  dependent.  During 
VALID,  FMECA' s  are  performed  at  the  functional  level  in  support  of 
reliability  modeling,  and  prediction  analyses.  During  FSED, 
detailed,  piece-part  level  FKECA's  are  the  norm.  Performance  of 
FMECA' s  during  PROD  is  considered  of  marginal  value  except  as 
performed  on  hardware  changes. 

Two  other  system  characteristics  were  found  to  affect  the  selection 
and  extent  of  application  of  the  FMECA  task.  During  FSED  and  on 
programs  involving  the  development  of  "single  thread"  systems,  the 
FMECA  task  was  often  deleted  for  cost  effectiveness  reasons.  It  was 
also  observed  that  programs  involving  integration  of  proven  hardware 
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had  little  requirement  for  piece-part  level  FMECA's.  Functional 
FMECA's  with  attendant  reduced  cotta  were  more  the  norm  for  these 
types  of  programs. 

SNEAK  CIRCUIT  ANALYSIS  (SCA) 

(Program  Category  %'s  Not  Applicable) 

The  sneak  circuit  analysis  task  was  determined  to  be  an  expensive 
task  and  only  selectively  applied.  In  the  survey  data  base  it  was 
applied  to  two  VALID  programs.  On  one  program  it  represented  1.5% 
of  the  engineering  budget  (11. 2%  of  the  reliability  budget)  while  on 
the  second  it  was  only  a  token  effort  with  expenditure  of  only  one 
manmonth.  A  partial  FMECA  was  integrated  with  the  former  effort. 

While  the  cost  data  was  sparse  on  this  task,  the  nature  of  the  task 
in  combination  with  its  cost  suggests  tailoring  in  the  form  of 
selective  application. 

ELECTRONIC  PARTS/CIRCUIT  TOLERANCE  ANALYSIS 

VALID  FSED  PROD  SMALL  SPACE 

5.9Z  3.1%  —  — 
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The  circuit  tolerance  analysis  task  was  veil  represented  in  the 
survey  data  base.  It  is  an  engineering  oriented  analysis  task  with 
minimum  monitoring  by  the  reliability  function.  Its  application  in 
the  data  base  was  uniform  and  no  specific  tailoring  was  observed. 


PARTS 

PROGRAM 

VALID 

FSED 

PROD 

SMALL  SPACE 

8.7% 

51% 

34.3% 

10.9% 

Parts  program  costs  were  found  to  be  significantly  program  and  phase 
dependent.  Costs  ranged  up  to  51%  of  the  total  reliability  program 
cost  during  FSED.  Included  in  one  program  effort  was  the  cost 
incurred  to  rescreen  class  "B"  parts  to  class  ”S".  This  rescreening 
activity  was  not  considered  as  a  normal  parts  program  activity. 
Because  of  its  cost  impact,  the  parts  task  is  a  prime  candidate  for 
tailoring.  However,  full  treatment  of  the  tailoring  possibilities 
is  beyond  the  scope  of  this  study. 


CRITICAL  ITEMS 

VALID 

FSED 

PROD  SMALL  SPACE 

2.2% 

.1% 

.1% 
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Little  cost  tailoring  opportunity  exists  for  this  task.  The  primary 
tailoring  observed  in  the  data  was  a  complete  deletion  of  the  task 
on  several  programs. 

EFFECTS  OF  TEST,  STORAGE  ETC. 

VALID  FSED  PROD  SMALL  SPACE 

6.0Z  .72  22 

There  were  seven  data  entries  on  this  task  including  two  from  more 
recent  VALID  phase  programs.  The  data  indicates  a  trend  towards 
increased  emphasis  on  this  task,  especially  on  those  systems  whose 
exposure  to  failure  during  storage  exeeds  that  of  the  operational 
mission.  On  the  two  more  recent  VALID  phase  programs  where  there 
was  a  "readiness”  reliability  requirement,  the  task  cost  averaged 
102  of  the  reliability  effort. 

The  regression  equations  confirmed  the  significance  of  this  task  in 
the  presence  of  a  readiness  requirement.  With  a  readiness 
requirement  present,  task  cost  is  in  the  neighborhood  of  .25%  of  the 
engineering  budget  range.  Without  a  readiness  requirement,  the  task 
cost  was  negligible. 


RELIABILITY  TESTING 


VALID 

FSED 

PROD 

SMALL  SPACE 

10.1% 

9.8% 

9.8% 

9.7% 

The  typical  reliability  testing  task  represents  10%  of  the 
reliability  effort,  independent  of  program  type  or  phase.  These 
costs  reflect  only  those  efforts  charged  directly  to  the  reliability 
function.  Included  are  charges  for  developing  reliability  test 
plans  and  procedures,  for  test  monitoring  and  for  test  reporting. 
Costs  associated  with  test  performance,  test  hardware,  technicians, 
etc.  are  charged  against  the  engineering  or  manufacturing  cost 
centers. 

Opportunity  for  tailoring  of  the  reliability  test  task  are 
considerable.  More  recent  programs  indicate  an  increased  emphasis 
on  combined  environment  reliability  testing  (CERT)  and  environmental 
stress  screening  (ESS)  with  a  reduced  emphasis  on  formal 
demonstration. 
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APPENDIX  A  -  DATA  BASE 


Quantitative  information  used  in  this  study  was  developed  from  data 
collected  from  twelve  separate  space  and/or  missile  programs.  One 
of  the  programs  yielded  data  for  both  an  FSED  and  a  PROD  phase  thus 
increasing  the  number  of  data  sets  to  thirteen.  The  programs 
selected  for  inclusion  in  this  study  range  in  type  from  small  space 
to  large  1CBM  programs. 

Table  A-l  lists  the  programs  by  their  types  and  gives  the  designator 
by  which  they  will  be  referred  to  in  this  appendix. 

Table  A-2  lists  the  reliability  tasks  and  the  assigned  reference 
numbers  used  throughout  this  appendix. 

The  raw  program  data  is  presented  in  Table  A-3.  This  tables  give 
the  actual  manmonth  expenditures  for  the  sample  programs.  Included 
in  the  table  are  engineering  support  (representing  the  effort 
contributed  to  the  reliability  program  by  other  engineering 
disciplines)  and  engineering  budget  (representing  the  total 
engineering  effort).  As  may  be  seen  from  the  raw  data  base,  the 
task  expenditures  for  a  program  are  highly  dependent  on  the  sice  of 
the  program;  that  is,  a  program  with  a  large  engineering  budget  has 
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PROGRAM 

DESIGNATION 


PROGRAM 

PEASE 

PROGRAM  TYPE 

PROGRAM 

DURATION 

(YEARS) 

VALID 

MISSILE/ SPACE 

5.5 

VALID 

SMALL  MISSILE 

2.5 

VALID 

MISSILE/TORPEDO 

3.25 

VALID 

MISSILE/SPACE 

4.0 

PSED 

AIR  TO  GROUND  MISSILE 

6.25 

FSED 

AIR  TO  GROUND  MISSILE 

2.25 

FSED 

LARGE  SPACE 

5.0 

FSED 

ICBM  (GROUND  SEGMENT) 

5.0 

PROD 

AIR  TO  GROUND  MISSILE 

3.0 

PROD 

ICBM  (FLIGHT  SEGMENT) 

4.0 

PROD 

GROUND  TO  AIR  MISSILE 
(GROUND  SEG) 

3.25 

PROD 

SMALL  SPACE 

4.25 

PROD 

SMALL  SPACE 

2.5 
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TABLE  A- 2 


PROGRAM  TASKS 


MIL-STD-785B 
TASK  NO. 

TASK  DESCRIPTION 

101 

PROGRAM  PLAN 

102 

MONITOR  &  CONTROL  SUBCONTRACTOR 

103 

PROGRAM  REVIEWS 

104  &  10S 

FRACAS  &  FRB 

201  -  203 

ANALYSIS 

204 

FMECA 

205 

SNEAK  CIRCUIT  ANALYSIS  (SCA) 

206 

ELEC.  PARTS/CIRC.  TOLERANCE  ANAL. 

207 

PARTS 

208 

CRITICAL  ITEMS 

209 

EFFECTS  OF  TEST,  STORAGE,  ETC. 

301  -  304 

TESTING 

ENG.  SUPPORT 
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more  dollars  to  spend  for  reliability  than  a  program  with  a  small 
engineering  budget.  For  the  purpose  of  comparison  it  is  useful  to 
compute  the  reliability  expenditures  as  a  percentage  of  the 
engineering  budget,  Table  A-4. 

A  set  of  characteristics  was  developed  which  represent  those 
features  believed  to  produce  the  greatest  influence  on  the 
allocation  of  time  to  do  a  given  task.  Table  A-S  lists  the  scoring 
criteria  for  the  characteristics  and  Table  A-6  gives  the  scoring  for 
each  of  the  programs  in  the  sample. 

For  the  multiple  regression  analysis  performed  in  Appendix  B,  a 
modified  version  of  the  data  set  presented  in  Table  A-4  is  used. 

The  modified  data  set.  Table  A-7,uses  a  subset  of  the  set  of  tasks 
and  purposely  leaves  out  those  values  which  are  considered  to  be 
nonrepresentative;  that  is,  those  values  which  were  strongly 
influenced  by  factors  outside  of  the  characteristic  set  and  which 
are  atypical.  Task  6  for  Program  A  has  been  listed  under  FSED 
because  it  was  performed  on  a  piece  part  level  which  is  not  typical 
of  a  VALID  program. 

In  Section  2.0  of  the  Overview,  a  set  of  averages  was  presented 
which  was  based  on  a  manmonth  per  year  level  of  effort.  Table  A-8 


TABLE  A-4  -  RELIABILITY  MANMOMTH  EXPENDITURE  AS  Z  OF  ENGINEERING  BUDGET 
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TABLE  A-5 


CHARACTERISTIC  SCORING  CRITERIA 


PROGRAM 

PHASE 


RELIABILITY 

CRITICALITY 


COMPLEXITY 


HARDWARE 

STATUS 


NUMBER  OF 
PART  TYPES 


MISSION 

REQUIREMENT 


REAOINESS 

REQUIREMENT 


PERCENTAGE  OF 
FAILURE  RATE 
SUBCONTRACTED 


CONFIGURATION 
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presents  the  program  data  in  the  manmonth  per  year  form  which  was 
used  to  develop  these  averages.  (The  data  in  Table  A-8  is  simply 
the  data  in  Table  A-3  divided  by  the  program  durations  listed  in 
Table  A-l) 

ACHIEVED  VERSUS  PREDICTED  RELIABILITY 

The  achieved  reliability  level  for  Programs  A,  L,  and  M  are  shown 
below  in  Table  A-9.  The  reliability  levels  for  Programs  E,  F,  I, 
and  J  are  classified;  however,  in  each  case  the  achieved  levels  were 
at  or  above  requirements.  The  remaining  programs  are  either  in  the 
development  stage  or  are  not  yet  in  operational  use,  thuB  no  data  is 
available  for  these  programs. 


TABLE  A-9  -  PREDICTED  VS  MEASURED  RELIABILITY 


PROGRAM 

DESIGNATOR 

PREDICTED 

RELIABILITY 

MEASURED 

RELIABILITY 

COMMENT 

A 

.90 

1.00 

6  FLIGHTS, 

6 

SUCCESSES 

L 

.87 

1.00 

3  FLIGHTS, 

3 

SUCCESSES 

M 

.97 

.96 

23  FLIGHTS, 

22 

SUCCESSES 
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APPENDIX  B  -  ANALYSIS 


In  attempting  to  develop  a  cost  estimation  model  which  would,  for  a 
given  set  of  program  characteristics,  give  a  rough  estimate  of  the 
amount  of  effort  to  be  expended  doing  a  given  reliability  task,  it 
is  desirable  to  keep  the  model  simple.  The  simplest  model  which 
would  allow  all  or  a  selected  subset  of  the  characteristics  to  exert 
some  influence  on  the  cost  is  a  multilinear  model;  that  is, 


5  '  bo  *  *)  Vj 


(B-l ) 


where  dependent  variable,  y,  denotes  the  cost  estimate,  and 
independent  variables,  x ^  ,  denote  the  various  characteristics 
involved.  The  b's  in  equation  are  unknown  at  this  stage. 


For  a  set  of  sample  programs,  let  the  known  program  expenditures  for 
a  given  reliability  task  be  denoted  by  y^.  Similarly,  let  the 
characteristics  which  have  bearing  on  this  cost  be  denoted  by  x^ 
where  i  refers  to  the  program.  The  estimate  of  the  cost  for  the  ith 


program  is  then  given  by: 


b  +  £  x  b 

o  j  ij  j 


(B-2) 


The  difference  between  the  actual  and  the  estimated  costs  is  given 


-»i  -»i  ■»!  -  (b»  *f  \3  y  (,,-3) 

It  becomes  convenient  at  this  point  to  set  x^  equal  to  1  for  all 
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values  of  i  (since  no  characteristics  have  been  assigned  to  an  index 
as  yet),  and  set  bQ  to  zero.  The  form  of  the  equation  is  not 
changed  in  that  bj  has  replaced  bQ  in  the  role  of  the  constant,  but 
the  equation  may  now  be  more  easily  expressed  as: 

4  ■  n  -  h  ■  h  -  5  Vi  •  <B"4) 

A  further  compaction  of  notation  takes  place  by  recognizing  this  as 

A 

a  vector  equation  with  Ay,  y,  y,  and  b  representing  vectors  and  X 
representing  a  matrix.  Equation  B-4  may  now  be  written  as: 

Ayay— y=y-Xb  (B~5) 

The  best  estimate  of  the  cost  then  occurs  when  the  length  of  the 
difference  vector,  that  is  the  difference  between  the  actual  costs 
vector  and  the  estimated  costs  vector,  is  at  a  minimum.  Minimizing 
with  respect  to  b  and  solving  for  b  yields; 

b  -  (XTX)_1  XTy,  (B-6) 

where  T  denotes  the  transpose  and  ^  denotes  the  inverse. 

Solutions  of  this  kind  are  called  multiple  linear  regressions.  For 
a  regression  to  be  meaningful,  the  size  of  the  sample,  n,  must 
exceed  the  number  of  parameters,  m,  to  be  estimated;  thus  it  is 
desirable  to  choose  that  set  of  independent  variables  (in  this  case 
characteristics)  which  minimize  the  unbiased  standard  error  of 
estimate  given  by: 

s  -^Ay .  A  y/(n-m)  .  (B-7) 


The  usual  procedure  is  to  use  a  stepwise  regression  routine.  In 


this  procedure  the  independent  variable  yielding  the  smallest  s  is 
the  first  to  be  place  in  the  equation.  In  the  next  step  that 
independent  variable  which  when  coupled  with  the  variable  chosen  in 
the  first  step  yields  the  smallest  s  is  chosen  as  the  second 
variable  to  be  included  in  the  equation.  This  procedure  continues 
until  the  standard  error  no  longer  continues  to  decrease.  A  problem 
with  this  procedure  is  that  it  doesn't  necessarily  yield  the  best 
solution  since  it  does  not  consider  all  possible  combinations  of 
independent  variables.  For  this  study  a  computer  program  was 
developed  which  checks  all  possible  combinations  of  up  to  three  out 
of  nine  independent  variables  (counting  the  constant  that  means  four 
estimated  parameters). 

The  data  used  in  this  analysis  is  as  presented  in  Appendix  A  in 
Tables  A-6  and  A-7  with  the  exception  that  configuration  was  left 

I 

out  and  an  attribute  called  phase  II  was  added  to  account  for  a 
strong  nonlinear  influence  due  to  program  phase.  A  more  rigorous 
investigation  would  use  a  phase-by-phase  analysis  of  the  data,  but 
the  number  of  programs  in  the  data  set  was  too  small  in  this  case  to 
allow  this. 

Table  B-l  presents  the  results  of  this  investigation  for  tftose  nine 
selected  reliability  tasks  using  one,  two,  and  three  independent 
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TABLE  B-l  -  RESULTS  OF  REGRESSION  ANALYSIS 


1.00 


variables.  The  correlation  coefficient,  r,  has  been  included  as  an 
indication  of  the  closeness  of  fit. 

From  the  table  it  can  be  seen  that  phase  is  the  major  contributor  to 
variations  in  program  task  costs.  However,  the  other  items  in  the 
table  are  not  all  as  intuition  would  lead  one  to  believe.  For 
example:  it  seems  logical  that  the  cost  for  analysis  should  increase 
with  reliability  criticality,  yet  the  analysis  shows  a  negative 
relationship.  This,  of  course,  could  be  due  to  changes  in  the 
engineering  budget  sensitive  to  the  same  characteristics  which 
offset  the  changes  in  the  reliability  budget.  Such  masking  of  the 
data  could  be  responsible  for  the  inconclusive  result  or  it  may  also 
be  that  there  were  insufficient  data  points  or  that  the  data  itself 
was  not  reflective  of  the  characteristics. 

Other  analysis  methods  were  investigated  and  applied  but  produced  no 
improvement  over  the  linear  regression  methods  discussed  above. 
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APPENDIX  C  -  RELIABILITY  TASK  DESCRIPTIONS 


RELIABILITY  TASK  DESCRIPTIONS  -  CONDENSED  FROM  MIL-STD-785B 

TASK  SECTION  100  -  PROGRAM  SURVEILLANCE  AND  CONTROL 

TASK 

101  RELIABILITY  PROGRAM  PLAN 

102  MONITOR/CONTROL  OF  SUBCONTRACTORS  AND  SUPPLIERS 

103  PROGRAM  REVIEWS 

104  FAILURE  REPORTING,  ANALYSIS,  AND  CORRECTION 
ACTION  SYSTEM  (FRACAS) 

105  FAILURE  REVIEW  BOARD  (FRB) 

TASK  SECTION  200  -  DESIGN  AND  EVALUATION 

TASK 

201  RELIABILITY  MODELING 

202  RELIABILITY  ALLOCATIONS 

203  RELIABILITY  PREDICTIONS 

204  FAILURE  MODES,  EFFECTS,  AND  CRITICALITY  ANALYSIS  (FMECA) 

205  SNEAK  CIRCUIT  ANALYSIS  (SCA) 

206  ELECTRONIC  PARTS/ CIRCUITS  TOLERANCE  ANALYSIS 

207  PARTS  PROGRAM 

208  RELIABILITY  CRITICAL  ITEMS 

209  EFFECTS  OF  FUNCTIONAL  TESTING,  STORAGE,  HANDLING, 
PACKAGING,  TRANSPORTATION,  AND  MAINTENANCE 
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TASK  SECTION  300  -  DEVELOPMENT  AND  PRODUCTION  TESTING 

TASK 

301  ENVIRONMENTAL  STRESS  SCREENING  (ESS) 

302  RELIABILITY  DEVELOPMENT/GROWTH  TEST  (RDGT)  PROGRAM 

303  RELIABILITY  QUALIFICATION  TEST  (RQT)  PROGRAM 

304  PRODUCTION  RELIABILITY  ACCEPTANCE  TEST  (PRAT)  PROGRAM 


TASK  101 


101.1 


101.2 

101.2. 


RELIABILITY  PROGRAM  PLAN 

PURPOSE.  The  purpose  of  Task  101  is  to  develop  a 
reliability  program  plan  which  identifies  and  ties 
together  all  program  management  tasks  required  to 
accomplish  program  requirements. 

TASK  DESCRIPTION. 

A  reliability  program  plan  shall  be  prepared  and  shall 
include,  but  not  be  limited  to,  the  following: 

a.  A  description  of  how  the  reliability  program  will  be 
conducted  to  meet  the  requirements  of  the  SOW. 

b.  A  detailed  description  of  how  each  specified  re¬ 
liability  accounting  and  engineering  design  task(s) 
will  be  performed  or  complied  with. 

c.  The  procedures  (wherever  existing  procedures  are 
applicable)  to  evaluate  the  status  and  control  of 
each  task,  and  identification  of  the  organisation 
unit  with  the  authority  and  responsibility  for 
executing  each  task. 

d.  Description  of  interrelationships  of  reliability 
tasks  and  activities  and  description  of  how 
reliability  tasks  will  interface  with  other  system 


105 


! 

\ 

« 

I 

J 

« 

I 


oriented  tasks.  The  description  shall  specifically 
include  the  procedures  to  be  employed  which  assure 
that  applicable  reliability  data  derived  from,  and 
traceable  to,  the  reliability  tasks  specified  are 
integrated  into  the  LSAP  and  reported  on 
appropriate  LSAK. 

e.  A  schedule  with  estimated  start  and  completion  points 
for  each  reliability  program  activity  or  task. 

f.  The  identification  of  known  reliability  problems  to 
be  solved,  an  assessment  of  the  impact  of  these 
problems  on  meeting  specified  requirements,  and 
the  proposed  solutions  or  the  proposed  plan  to 
solve  these  problems. 

g.  The  procedures  or  methods  (if  procedures  do  not 
exist)  for  recording  the  status  of  actions  to 
resolve  problems. 

h.  The  designation  of  reliability  milestones  (includes 
design  and  test). 

i.  The  method  by  which  the  reliability  requirements  are 
disseminated  to  designers  and  associated  personnel. 

j.  Identification  of  key  personnel  for  managing  the 
reliability  program. 

k.  Description  of  the  management  structure,  including 
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interrelationship  between  line,  service,  staff,  and 
policy  organizations. 


l.  Statement  of  what  source  of  reliability  design  guide 
lines  or  reliability  design  review  checklist  will  be 
utilized. 

m.  Description  of  how  reliability  contributes  to  the 
total  design,  and  the  level  of  authority  and 
constraints  on  this  engineering  discipline. 

n.  Identification  of  inputs  that  the  contractor  needs 
from  operation  and  support  experience  with  a 
predecessor  item  or  items.  Inputs  should  include 
measured  basic  reliability  and  mission  reliability 
values,  measured  environmental  stresses,  typical 
failure  modes,  and  critical  failure  modes. 

101.2.2  The  contractor  may  propose  additional  tasks  or 
modifications  with  supporting  rationale  for  such 
additions  or  modifications. 

101.2.3  When  approved  by  the  procuring  activity  and  if 
incorporated  into  the  contract,  the  reliability 
program  plan  shall  become,  together  with  the  SOW, 
the  basis  for  contractual  compliance. 


TASK  102 


r-  -  ~i 

4 


MONITOR/CONTROL  OF  SUBCONTRACTORS  AND  SUPPLIERS 

102.1  PURPOSE.  The  purpose  of  Task  102  is  to  provide  the 
prime  contractor  and  PA  with  appropriate  surveillance 
and  management  control  of  subcontractors/suppliers 
reliability  programs  so  that  timely  management  action 
can  be  taken  as  the  need  arises  and  program  progress 
is  ascertained. 

102.2  TASK  DESCRIPTION 

102.2.1  The  contractor  shall  insure  that  system  elements 
obtained  from  suppliers  will  meet  reliability 
requirements.  This  effort  shall  apply  to  CFE  items 
obtained  from  any  supplier  whether  in  the  first  or  any 
subsequent  tier,  or  whether  the  item  is  obtained  bv  »•#> 
intra-company  order  from  any  element  of  the  contractor's 
organization.  All  subcontracts  shall  include 
provisions  for  review  and  evaluation  of  the  suppliers' 
reliability  efforts  by  the  prime  contractor,  and  by  the 
procuring  activity  at  their  discretion. 

102.2.2  The  contractor  shall  assure,  and  advise  the  PA,  that  his 
subcontractor'  and  suppliers'  reliability  efforts  are 
consistent  with  overall  system  requirements,  and  that 
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provisions  are  made  for  surveillance  of  their 

reliability  activities.  The  contractor  shall,  as 

appropriate : 

a.  Incorporate  quantitative  reliability  requirements 
in  subcontracted  equipment  specifications. 

b.  Assure  that  subcontractors  have  a  reliability 
program  that  is  compatible  with  the  overall  program 
and  includes  provisions  to  review  and  evaluate 
their  supplier(s)  reliability  efforts. 

c.  Attend  and  participate  in  subcontractors'  design 
reviews . 

d.  Review  subcontractors'  predictions  and  analyses  for 
accuracy  and  correctness  of  approach. 

e.  Furnish  subcontractors  with  data  from  testing  or 
usage  of  their  product  when  testing  and  usage  are 
outside  their  control. 

f.  Review  subcontractors'  test  plans,  procedures,  and 
reports  for  correctness  of  approach  and  test  details. 

g.  Review  subcontractors'  progress  reports. 

h.  Assure  that  subcontractors  have,  and  are  pursuing,  a 
vigorous  corrective  action  effort  to  eliminate  causes 
of  unreliability. 


i.  Reserve  for  himself  and  for  the  FA  the  right  to  send 


personnel  into  the  subcontractors'  facilities  as 
necessary  to  monitor  and  evaluate  the  subcontractors' 
reliability  programs  and  related  activities. 

j.  Assure  that  subcontractors/suppliers  will  provide 
him  with  the  necessary  technical  and  administrative 
support  for  the  items  they  supply  during  production 
and  deployment  of  the  hardware.  This  support  may 
include  failure  analyses  and  corrective  action  for 
failures  occurring  in  the  total  use  environment, 

if  specified  under  102.2  herein. 

k.  Ensure  that  selected  items  (critical  items,  et 
cetera)  obtained  from  suppliers  are  covered  by 
specifications,  drawings,  and  other  technical 
documents  and  that  the  requirements  called  out 
adequately  control  those  parameters  and 
characteristics  that  may  affect  reliability  of  the 
end  item. 

l.  Unless  otherwise  specified  by  the  FA,  conduct  or 
control  his  subcontractors/ suppliers  reliability 
demonstration  (qualification  and  acceptance)  tests 
on  behalf  of  the  government  to  provide  a  defensible 
basis  for  determining  the  supplier's  contractual 
compliance  with  quantitative  reliability  requirement. 
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TASK  103 


PROGRAM  REVIEWS 


103.1  PURPOSE.  The  purpose  of  Task  103  is  to  establish  a 
requirement  for  the  prime  (or  associate)  contractor  to 
conduct  reliability  program  reviews  at  specified  points 
in  time  to  assure  that  the  reliability  program  is 
proceeding  in  accordance  with  the  contractual  milestones 
and  that  the  weapon  system,  subsystem,  equipment,  or 
component  quantitative  reliability  requirements  will 

be  achieved . 

103.2  TASK  DESCRIPTION 

103.2.1  The  reliability  program  shall  be  planned  and  scheduled 
to  permit  the  contractor  and  the  PA  to  review  program 
status.  Formal  review  and  assessment  of  contract 
reliability  requirements  shall  be  conducted  at  major 
program  points,  identified  as  system  program  reviews, 
as  specified  by  the  contract.  As  the  program  develops, 
reliability  progress  shall  also  be  assessed  by  the  use 
of  additional  reliability  program  reviews  as  necessary. 
The  contractor  shall  schedule  reviews  as  appropriate 
with  his  subcontractors  and  suppliers  and  insure  that 
the  PA  is  informed  in  advance  of  each  review. 
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The  reviews  shell  identify  and  discuss  all  pertinent 
aspects  of  the  reliability  program  such  as  the 
following,  when  applicable: 

a.  AT  THE  PRELIMINARY  DESIGN  REVIEW  (PDR): 

(1)  Updated  reliability  status  including: 

(a)  Reliability  modeling 

(b)  Reliability  apportionment 

(c)  Reliability  predictions 

(d)  FMECA 

(e)  Reliability  content  of  specification 

(f)  Design  guideline  criteria 

(g)  Other  tasks  as  identified 

(2)  Other  problems  affecting  reliability 

(3)  Parts  program  progress 

(4)  Reliability  critical  items  program 

b.  AT  THE  CRITICAL  DESIGN  REVIEW  (CDR) : 

(1)  Reliability  content  of  specifications 

(2)  Reliability  prediction  and  analyses 

(3)  Parts  program  status 

(4)  Reliability  critical  items  program 

(5)  Other  problems  affecting  reliability 

(6)  FMECA 


(7)  Identification  of  circuit  reference  designators 
whose  stress  levels  exceed  the  recommended 
parts  application  criteria. 

(8)  Other  tasks  as  identified 

c.  AT  RELIABILITY  PROGRAM  REVIEWS: 

(1)  Discussion  of  those  items  reviewed  at  PDRs  and 
CDRs 

(2)  Results  of  failure  analyses 

(3)  Test  schedule:  start  dates  and  completion 
dates 

(4)  Parts,  design,  reliability,  and  schedule 
problems 

(5)  Status  of  assigned  action  items 

(6)  Contractor  assessment  of  reliability  task 
effectiveness 

(7)  Other  topics  and  issues  as  deemed  appropriate 
by  the  contractor  and  the  PA 

d.  AT  THE  TEST  READINESS  REVIEW: 

(1)  Reliability  analyses  status,  primary  prediction 

(2)  Test  schedule 

(3)  Test  profile 

(4)  Test  plan  including  failure  definition 
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(5)  Test  report  format 

(6)  FRACAS  implementation 


AT  THE  PRODUCTION  READINESS  REVIEW: 

(1)  Results  of  applicable  RQT's 

(2)  Results  of  applicable  reliability/growth 
testing 


TASK  104 


FAILURE  REPORTING,  ANALYSIS,  AND  CORRECTIVE  ACTION  SYSTEM  (FRACAS) 

104.1  PURPOSE.  The  purpose  of  Task  104  is  Co  establish  a 
closed  loop  failure  reporting  system,  procedures  for 
analysis  of  failures  to  determine  cause,  and 
documentation  for  recording  corrective  action  taken. 

104.2  TASK  DESCRIPTION 

104.2.1  The  contractor  shall  have  a  closed  loop  system  that 

collects,  analyzes,  and  records  failures  that  occur 
for  specified  levels  of  assembly  prior  to  acceptance  of 
the  hardware  by  the  procuring  activity.  The 
contractor's  existing  data  collection,  analysis  and 
corrective  action  system  shall  be  utilized,  with 
modification  only  as  necessary  to  meet  the  requirements 
specified  by  the  FA. 

104.2.2  Procedures  for  initiating  failure  reports,  the  analysis 

of  failures,  feedback  of  corrective  action  into  the 
design,  manufacturing  and  test  processes  shall  be 
identified.  Flow  diagram(s)  depicting  failed  hardware 
and  data  flow  shall  also  be  documented.  The  analysis 


of  failures  shall  establish  and  categorize  the  cause 


of  failure. 
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104.2.3 


104.2.4 


104.2.5 


The  closed  loop  system  shall  include  provisions  to 
assure  that  effective  corrective  actions  are  t3ker.  on 
a  timely  basis  by  a  follow-up  audit  that  reviews  all 
open  failure  reports,  failure  analyses,  and  corrective 
action  suspense  dates,  and  the  reporting  of 
delinquencies  to  management.  The  failure  cause  for 
each  failure  shall  be  clearly  stated. 

V.'hen  applicable,  the  method  of  establishing  and 
recording  operating  time,  or  cycles,  on  equipments 
shall  be  clearly  defined. 

The  contractor's  closed  loop  failure  reporting  system 
data  shall  be  transcribed  to  Government  forms  only  if 
specifically  required  by  the  procuring  activity. 
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TASK  105 


FAILURE  REVIEW  EOARD  (FRB) 

105.1  PURPOSE.  The  purpose  of  Task  105  is  to  require  the 
establishment  of  a  failure  review  board  to  review 
failure  trends,  significant  failures,  corrective  actions 
status,  and  to  assure  that  adequate  corrective  actions 
are  taken  in  a  timely  manner  and  recorded  during  the 
development  and  production  phases  of  the  program. 

105.2  TASK  DESCRIPTION 

105.2.1  The  FRB  shall  review  functional/performance  failure 

data  from  appropriate  inspections  and  testing  including 
subcontractor  qualification,  reliability,  and  acceptance 
test  failures.  All  failure  occurrence  information  shall 
be  available  to  the  FRB.  Data  including  a  description 
of  test  conditions  at  time  of  failure,  symptoms  of 
failure,  failure  isolation  procedures,  and  known  or 
suspected  causes  of  failure  shall  be  examined  by  the 
FRB.  Open  FRB  items  shall  be  followed  up  until 
failure  mechanisms  have  been  satisfactorily 
identified  and  corrective  action  initiated.  The 
FRB  shall  also  maintain  and  disseminate  the  status 
of  corrective  action  implementation  and 
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effectiveness.  Minutes  of  FRB  activity  shall  be 
recorded  and  kept  on  file  for  examination  by  the 
examination  by  the  procuring  activity  during  the  term 
of  the  contract.  Contractor  FRB  members  shall  include 
appropriate  representatives  from  design,  reliability, 
system  safety,  maintainability,  manufacturing,  and 
parts  and  quality  assurance  activities.  The  procuring 
activity  reserves  the  right  to  appoint  a  representative 
to  the  FRB  as  an  observer.  If  the  contractor  can 
identify  and  utilize  an  already  existing  and  operating 
function  for  this  task,  then  he  shall  describe  in  his 
proposal  how  that  function  will  be  employed  to  meet  the 
procuring  activity  requirements.  This  task  shall  be 
coordinated  with  Quality  Assurance  organizations  to 
insure  there  is  no  duplication  of  effort. 


TASK  201 


RELIABILITY  MODELING 


PURPOSE.  The  purpose  of  Task  201  is  to  develop  a 
reliability  model  for  making  numerical  apportionments 
and  estimates  to  evaluate  system/ subsystem/ equipment 
reliability. 

TASK  DESCRIPTION 

A  reliability  mathematical  model  based  on  system/sub- 
sy stem/equipment  functions  shall  be  developed  and 
maintained.  As  the  design  evolves,  a  reliability 
block  diagram  shall  be  developed  and  maintained  for 
the  system/ subsystem  with  associated  allocations  and 
predictions  for  all  items  in  each  reliability  block. 
The  reliability  block  diagram  shall  be  keyed  and 
traceable  to  the  functional  block  diagram,  schematics, 
and  drawings,  and  shall  provide  the  basis  for  accurate 
mathematical  representation  of  reliability. 
Nomenclature  of  items  used  in  reliability  block 
diagrams  shall  be  consistent  with  that  used  in 
functional  block  diagrams,  drawings,  and  schematics, 
weight  statements,  power  budgets,  and  specifications. 
The  model  outputs  shall  be  expressed  in  terms  of 
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201.2. 


201.2.3 


contractual  reliability  requirements  and  other 
reliability  terms  as  specified,  l.'hen  required  for  the 
PROD  phase,  the  model  shall  be  updated  to  include 
hardware  design  changes. 

The  reliability  mathematical  model  shall  be  updated 
with  information  resulting  from  reliability  and  other 
relevant  tests  as  well  as  changes  in  item  configuration, 
mission  parameters  and  operational  constraints.  Inputs 
and  outputs  of  the  reliability  mathematical  model  shall 
be  compatible  with  the  input  and  output  requirements 
of  the  system  and  subsystem  level  analysis  models. 
Modeling  techniques  shall  provide  separate  outputs  for: 
(1)  basic  reliability,  and  (2)  mission  reliability,  of 
the  system/ sub-system/ equipment .  A  single  series 
calculation  of  basic  reliability,  and  the  modeling 
techniques  described  in  Appendix  A  of  MIL-HDBK-217  for 
mission  reliability,  shall  be  used  unless  otherwise 
specified. 
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TASK  202 

RELIABILITY  ALLOCATIONS 


202.1  PURPOSE.  The  purpose  of  Task  202  is  to  assure  that 
once  quantitative  system  requirements  have  been 
determined,  they  are  allocated  or  apportioned  to 
lower  levels. 

202.2  TASK  DESCRIPTION 

202.2.1  Both  basic  reliability  and  mission  reliability 

requirements  shall  be  allocated  to  the  level  specified 
and  shall  be  used  to  establish  baseline  requirements 
for  designers.  Requirements  consistent  with  the 
allocations  shall  be  imposed  on  the  subcontractors  and 
suppliers.  The  apportioned  values  shall  be  included 
in  appropriate  sections  of  procurement  specifications, 
critical  item  specifications,  and  contract  end  item 
specifications.  All  allocated  reliability  values 
established  by  the  contractor  and  included  in  contract 
end  item  specifications  shall  be  consistent  with  the 
reliability  model  (see  Task  201)  and  any  change  thereto, 
and  subject  to  procuring  activity  review. 


-‘M 
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TASK  203 


203.1 


203.2 

203.2.1 


RELIABILITY  PREDICTIONS 


PURPOSE.  The  purpose  of  Task  203  is  to  estimate  the 
basic  reliability  and  mission  reliability  of  the 
system/ subsystem/equipment  and  to  make  a  determination 
of  whether  those  reliability  requirements  can  be 
achieved  with  the  proposed  design. 

TASK  DESCRIPTION 

Reliability  predictions  shall  be  made  for  the  system, 
subsystem/equipment .  When  required,  predictions  shall 
account  for,  and  differentiate  between,  each  mode  of 
item  operation  as  defined  in  the  item  specification. 
Predictions  shall  be  made  showing:  (1)  basic 
reliability  of  the  item  during  the  life  profile 
specified  by  the  PA,  to  provide  a  basis  for  life  cycle 
cost  and  logistics  support  analysis;  and  (2)  mission 
reliability  of  the  item  during  the  mission  profile(s) 
specified  by  the  PA,  to  provide  a  basis  for  analysis 
of  item  operational  effectiveness.  These  predictions 
shall  be  made  using  the  associated  reliability  block 
diagram  and  failure  rate  data  approved  by,  or  provided 
by,  the  procuring  activity.  Items  shall  not  be 
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203.2.1 


203.2.1 


203.2.1 


203.2. 


excluded  from  the  MCSP  or  other  mission  reliability 
predictions  unless  substantiating  documentation 
(such  as  FMECA)  verify  that  the  item  failure  has  no 
influence  on  the  required  measure  of  mission 
reliability.  Prior  to  such  exclusions  from  the 
predictions,  an  assessment  and  approval  shall  be 
obtained  from  the  procuring  activity. 

.1  Failure  rates  other  than  those  established  at  contract 
award  may  be  used  only  upon  approval  of  the  procuring 
activity. 

.2  The  permissible  failure  rate  adjustment  factors  for 
standby  operation  and  storage  shall  be  as  specifically 
agreed  to  by  the  procuring  activity. 

.3  When  the  individual  part  operating  conditions  are 
defined,  the  prediction  procedure  in  Section  2  of 
MIL-HDBK-217 ,  or  PA  approved  alternative,  shall  be  used. 
Predictions  for  electronic  equipment  shall  be  made 
using  one  of  the  two  methods  contained  in  MIL-HDBK-217 , 
or  alternatives  approved  or  provided  by  the  PA. 
Predictions  for  mechanical,  electrical,  and 
electro-mechanical  equipment  shall  be  made  using  either 
contractor  data  or  alternatives,  both  of  which  shall 
require  PA  approval. 
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TASK  204 

FAILURE  LODES,  EFFECTS,  AND  CRITICALITY  ANALYSIS  (FI.ECA) 

204.1  PURPOSE.  The  purpose  of  Task  204  is  to  identify 
potential  design  weaknesses  through  systematic, 
documented  consideration  of  the  following:  all  likely 
ways  in  which  a  component  or  equipment  can  fail;  causes 
for  each  node;  and  the  effects  of  each  failure  (which 
nay  be  different  for  each  mission  phase). 

204.2  TASK  DESCRIPTION 

204.2.1  FKECA  shall  be  performed  to  the  level  specified 

(subsystem,  equipment,  functional  circuit,  module,  or 
piece  part  level).  All  failure  modes  shall  be 
postulated  at  that  level  and  the  effects  on  all  higher 
levels  shall  be  determined.  The  FMECA  shall  consider 
failure  mode,  failure  effect  and  criticality  (impact  on 
safety,  readiness,  mission  success,  and  demand  for 
maintenance/logistics  support),  and  the  failure 
indication  to  the  operator  and  maintenance  personnel 
by  life/mission  profile  phase.  This  analysis  shall  be 
scheduled  and  completed  concurrently  with  the  design 
effort  so  that  the  design  will  reflect  analysis 
conclusions  and  recommendations.  The  results  and 
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current  status  of  FliECA  shall  De  used  as  inputs  to 
design  trade-offs,  safety  engineering,  maintenance 
engineering,  maintainability,  logistic  support  analysis, 
test  equipment  design  and  test  planning  activities,  et 
cetera. 

204.2.2  A  sample  FliECA  worksheet  format  shall  be  submitted  to 
the  PA  for  approval  and  details  such  as  who  (by 
discipline)  shall  perform  the  analysis,  who  shall 
review  it  for  adequancy  and  accuracy,  when  and  how  it 
shall  be  updated,  and  what  specific  uses  shall  be  made 
of  the  results  (e.g.,  identifying  potential  system 
weaknesses,  as  a  tool  for  evaluating  the  effectiveness 
of  built-in  test,  updating  reliability  assessments, 
updating  critical  item  control  procedures,  development 
of  safety,  maintainability,  and  human  engineering 
design  and  operational  criteria,  et  cetra)  shall  be 
identified. 
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TASK  205 


I 

1 


SNEAK  CIRCUIT  ANALYSIS  (SCA) 

205.1  PURPOSE.  The  purpose  of  Task  205  is  to  identify  latent 
paths  which  cause  occurrence  of  unwanted  functions  or 
inhibit  desired  functions,  assuming  all  components  are 
functioning  properly. 

205.2  TASK  DESCRIPTION 

205.2.1  Sneak  circuit  analyses  of  critical  circuitry  shall  be 
conducted  to  identify  latent  paths  which  cause  unwanted 
functions  to  occur  or  which  inhibit  desired  functions. 
In  making  these  analyses,  all  components  shall  be 
assumed  to  be  functioning  properly.  These  analyses 
shall  be  made  using  production  manufacturing 
documentation  for  each  circuit  analyzed. 

205.2.2  A  list  of  those  functions/circuits  to  be  analyzed, 
and  the  priorities  given  each  subassembly  in  the 
analysis,  shall  be  percented  for  AP  approval  at  CDR, 
together  with  the  supporting  rationale  for  the 
selections  made.  Results  of  the  analyses  and  actions 
taken  as  a  result  of  analyses  findings  shall  be  made 
available  to  the  procuring  activity  upon  request. 


TASK  2 06 


ELECTRONIC  PARTS/CIRCUITS  TOLERANCE  ANALYSIS 

206.1  PURPOSE.  The  purpose  of  Task  206  is  to  examine  the 
effects  of  parts/circuits  electrical  tolerances  and 
parasitic  parameters  over  the  range  of  specified 
operating  temperatures. 

206.2  TASK  DESCRIPTION 

206.2.1  Parts/circuits  tolerance  analyses  shall  be  conducted  on 

critical  circuitry  as  defined  in  the  contract.  These 
analyses  shall  verify  that,  given  reasonable 
combinations  of  vithin-specification  characteristics 
and  parts  tolerance  buildup,  the  circuitry  being 
analyzed  will  perform  within  specification  performance. 
In  making  these  analyses  the  contractor  shall  examine 
the  effect  of  component  parasitic  parameters,  input 
signal  and  power  tolerances,  and  impedance  tolerances 
on  electrical  parameters,  both  at  circuit  nodes 
(component  interconnections)  and  at  input  and  output 
points.  Since  all  of  the  stated  factors  may  not  be 
significant  to  all  circuits,  only  the  critical  facto  < 
for  that  circuit  shall  be  considered. 
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206.2.2 


Component  characteristics,  (life-drift  anc!  temperature) 
shall  be  factored  into  the  analyses.  These 
characteristics  or  values  shall  include  resistance, 
capacitance,  transistor  gain,  relay  opening  or  closing 
time,  et  cetera. 

206.2.3  The  inductance  of  wire-wound  resistors,  parasitic 
capacitance,  and  any  other  similar  phenomena  shall  be 
taken  into  account,  where  appropriate.  Maximum 
variations  in  input  signal  or  power  supply  voltage, 
frequency,  bandwidth,  impedance,  phase,  et  cetera  shall 
be  used  in  the  analyses.  The  impedance  characteristics 
of  the  load  shall  be  considered  as  well.  Circuit  node 
parameters  (including  voltage,  current,  phase,  and 
waveform),  circuit  element  rise  time,  timing  of 
sequential  events,  circuit  power  dissipation,  and 
circuit-load  impedance  matching  under  worst  case 
conditions  shall  also  be  considered.  These  parameters 
shall  be  analyzed  for  their  effect  on  the  performance 
of  circuit  components. 

206.2.4  A  list  of  those  functions/circuits  to  be  analyzed  shall 
be  presented  at  FDR.  The  most  infavorable  combination 
of  realizable  conditions  to  be  considered  in  the 
parts/circuits  tolerance  analyses  shall  be  defined  for 
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approval  by  the  procuring  activity.  Results  of  the 
analyses  and  actions  taken  as  a  result  of  analyses 


finainfas  shall  be  made  available  to  the  procuring 


activity  upon  request. 


TASK  207 


207.1 

207.2 

207.2.1 

207.2.2 


PARTS  PROGRAM 


PURPOSE.  The  purpose  of  Task  207  is  to  control  the 
selection  and  use  of  standard  and  nonstandard  parts. 
TASK  DESCRIPTION 

A  parts  control  program  shall  be  established  in 
accordance  with  MIL-STD-965  procedures,  as  designated 
in  the  contract. 

Reliability  design  guidelines  shall  be  developed  and 
documented  to  include  derating  criteria,  junction 
temperatures,  and  parts  application  criteria.  Safety 
margins  for  nonelectronic  parts  will  also  be  included 
when  appropriate.  The  guidelines  shall  be  consistent 
with  guidance  provided  by  the  PA. 
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TASK  208 


RELIABILITY  CRITICAL  ITEMS 


208.1  PURPOSE.  The  purpose  of  Task  208  is  Co  identify  and 
control  these  items  which  require  "special  attention" 
because  of  complexity,  application  of  advanced 
state-of-the-art  techniques,  and  the  impact  of 
potential  failure  on  safety,  readiness,  mission 
success,  and  demand  for  maintenance/logistics  support. 

208.2  TASK  DESCRIPTION 

208.2.1  Reliability  critical  items  shall  be  identified  by  FMECA 
or  other  methods  and  shall  be  controlled.  Methods  and 
procedures  for  control  and  testing  of  the  reliability 
critical  items  shall  be  identified  along  with 
justif ication(s)  for  decontrolling  the  item  if  that  is 
intended.  When  specified,  the  procedures  shall  include 
engineering  support  of  critical  items  during  FSED 
government  field  testing,  which  shall  include 
provisions  for  confirming  failures  which  may  occur, 
expediting  failure  cause  determination,  and  determining 
and  incorporating,  or  verifying,  the  necessary 
corrective  action. 
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TASK  209 


i 


EFFECTS  OF  FUNCTIONAL  TESTING ,  STORAGE,  UAKLLIKG , 

PACKAGING  ,  TRANSPORTATION ,  ALT  ilAIUTEKANCE 

209.1  PURPOSE.  The  purpose  of  Task  209  is  to  determine  the 
effects  of  storage,  handling,  packaging,  transportation, 
maintenance,  and  repeated  exposure  to  functional 
testing  on  hardware  reliability. 

209.2  TASK  DESCRIPTION 

209.2.1  Procedures  shall  be  established,  maintained,  and 

implemented  to  determine  by  test  and  analysis,  or 
estimation,  the  effects  of  storage,  handling, 
packaging,  transportation,  maintenance,  and  repeated 
exposure  to  functional  testing  on  the  design  and 
reliability  of  the  hardware.  The  results  of  this 
effort  shall  include  items  such  as: 
a.  Identification  of  equipments  and  their  major  or 
critical  characteristics  which  deteriorate  with 
storage  age  or  environmental  conditions 
(including  shock  and  vibration,  et  cetera). 
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j.  Iaeut  ificat  ion  uf  procedures  for  periodic  field 
inspection  or  tests  (including  recall  for  test) 
or  stockpile  reliability  evaluation.  The 
procedures  shall  include  suggested  quantity  of 
items  for  test  and  acceptable  levels  of 
performance  for  parameters  under  test, 
c.  Identification  of  special  procedures  for 
maintenance  or  restoration. 

The  results  of  this  effort  shall  be  used  to  support 
lonD  term  failure  rate  predictions,  design  trade-offs, 
definition  of  allowable  test  exposures,  retest  after 
storage  decisions,  packaging,  handling,  or  storage 
requirements,  and  refurbishment  plans. 


TASK  301 


ENVIRONMENTAL  STRESS  SCREENING  (ESS) 

301.1  PURPOSE.  The  purpose  of  Task  301  is  to  establish  and 
implement  environmental  stress  screening  procedures  so 
that  early  failure(s)  due  to  weak  parts,  workmanship 
defects,  and  other  non-conformance  anomolies  can  be 
identified  and  removed  from  the  equipment. 

301.2  TASK  DESCRIPTION 

301.2.1  Environmental  stress  screening  (also  known  as  pre¬ 
conditioning,  burn-in,  et  cetera)  shall  be  conducted 
on  parts,  subassemblies,  and  complete  units  for  both 
developmental  and  production  items. 

301.2.1.1  During  development,  ESS  test  procedures,  taking  into 
consideration  the  equipment  design,  part /component 
technology,  and  production  fabrication  techniques, 
shall  be  formulated.  ESS  procedures  shall  be  designed 
for  the  end  item  and  for  all  lower  level  items  which 
will  be  procured  separately  as  spare  or  repair  parts. 

A  plan  for  implementing  these  procedures  shall  also  be 
prepared,  indicating  the  proposed  application  of  ESS 
during  development  and  production.  The  proposed  ESS 
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procedures  and  implementation  plan  shall  be  subject  to 
approval  by  the  PA. 

301.2.2  ESS  testing  shall  be  designed  to  stimulate  relevant 
failures  by  stressing  the  item.  The  stressing  need  not 
simulate  the  precise  operational  environment  the  item 
will  see.  Environmental  stress  types  may  be  applied 

in  sequence.  During  ESS,  the  item  shall  be  cycled 
through  its  operational  modes  while  simultaneously 
being  subjected  to  the  required  environmental  stresses. 

301.2.3  Upon  approval  of  the  proposed  ESS  procedures  and 
implementation  plan,  a  detailed  environmental  stress 
screening  test  plan  shall  be  prepared  and  included  as 
part  of  the  reliability  test  plan.  The  ESS  detailed 
test  plan  shall  include  the  following,  subject  to  PA 
approval  prior  to  initiation  of  testing: 

a.  Description  of  environmental  stress  types,  levels, 
profiles,  and  exposure  times  to  be  applied. 

b.  Identification  of  level  (board,  sub-assembly, 
assembly)  at  which  testing  will  be  accomplished. 

c.  Identification  of  item  performance  and  stress 
parameters  to  be  monitored  during  ESS. 

d.  Proposed  test  duration  (failure-free  interval  and 
maximum  ESS  test  time  per  item). 
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301.2.4 


The  results  of  ESS  testing  during  development  shall  be 
analyzed  and  used  as  the  basis  for  the  ESS  procedures 
to  be  specified  for  production. 


;  ! 
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TASK  302 


RELIABILITY  DEVELOPMENT /GROWTH  TEST  (RDGT)  PROGRAM 

302.1  PURPOSE.  The  purpose  of  Task  302  is  to  conduct  pre¬ 
qualification  testing  (also  known  as  TAAF)  to  provide 
a  basis  for  resolving  the  majority  of  reliability 
problems  early  in  the  development  phase,  and 
incorporating  corrective  action  to  preclude  recurrence, 
prior  to  the  start  of  production. 

302.2  TASK  DESCRIPTION 

302.2.1  A  reliability  development /growth  test  (TAAF  test)  shall 
be  conducted  for  the  purpose  of  enhancing  system 
reliability  through  the  identification,  analysis,  and 
correction  of  failures  and  the  verification  of  the 
corrective  action  effectiveness.  Mere  repair  of  the 
test  item  does  not  constitute  corrective  action. 

302.2.1.1  To  enhance  mission  reliability,  corrective  action  shall 
be  focused  on  mission-critical  failure  modes.  To 
enhance  basic  reliability,  corrective  action  shall  be 
focused  on  the  most  frequent  failure  modes  regardless 
of  their  mission  criticality.  These  efforts  shall  be 
balanced  to  meet  predicted  growth  for  both  parameters. 

302.2.1.2  Growth  testing  will  emphasize  performance  monitoring. 


failure  detection,  failure  analysis,  and  the  in¬ 
corporation  and  verification  of  design  corrections  to 
prevent  recurrence  of  failures. 

302.2.2  A  TAAF  test  plan  shall  be  prepared  and  shall  include 
the  following  subject  to  PA  approval  prior  to 
initiating  of  testing: 

a.  Test  objectives  and  requirements,  including  the 
selected  growth  model  and  growth  rate  and  the 
rationale  for  both  selections. 

b.  Identification  of  the  equipment  to  be  tested  and 
the  number  of  test  items  of  each  equipment. 

c.  Test  conditions,  environmental,  operational  and 
performance  profiles,  and  the  duty  cycle. 

d.  Test  schedules  expressed  in  calendar  time  and  item 
life  units,  including  the  test  milestones  and  test 
program  review  schedule. 

e.  Test  ground  rules,  chargeability  criteria  and 
interface  boundaries. 

f.  Test  facility  and  equipment  descriptions  and 
requirements. 

g.  Procedures  and  timing  for  corrective  actions. 
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h.  Blocks  of  time  aad  resources  designated  for  the 


incorporation  of  design  corrections. 

i.  Data  collection  and  recording  requirements. 

j.  FRACAS. 

k.  Government  furnished  property  requirements. 

l.  Description  of  preventive  maintenance  to  be 
accomplished  during  test. 

m.  Final  disposition  of  test  items. 

n.  Any  other  relevant  considerations. 

302.2.3  As  specified  by  the  procuring  activity,  the  TAAF  test 
plan  shall  be  submitted  to  the  procuring  activity  for 
its  review  and  approval.  This  plan,  as  approved, 
shall  be  incorporated  into  the  contract  and  shall 
become  the  basis  for  contractual  compliance. 
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TASK  303 

RELIABILITY  QUALIFICATION  TEST  ( RQT )  PR0GRA11 

303.1  PURPOSE.  The  purpose  of  Task  303  is  to  determine  that 
the  specified  reliability  requirement  have  been 
achieved . 

303.2  TASK  DESCRIPTION 

303.2.1  Reliability  qualification  tests  shall  be  conducted  on 
equipments  which  shall  be  identified  by  the  PA  and 
which  shall  be  respresentat ive  of  the  approval 
production  configuration.  The  reliability  qualification 
testing  may  be  integrated  with  the  overall  system/ 
equipment  qualification  testing,  when  practicable,  for 
cost-effectiveness;  the  RQT  plan  shall  so  indicate  in 
this  case.  The  PA  shall  retain  the  right  to  disapprove 
the  test  failure  relevancy  and  chargeability 
determinations  for  the  reliability  demonstrations. 

303.2.2  A  RQT  plan  shall  be  prepared  in  accordance  with  the 
requirements  of  MIL-STD-781,  or  alternative  approved 
by  the  PA,  and  shall  include  the  following,  subject  to 
PA  approval  prior  to  initiation  of  testing: 

a.  Test  objectives  and  selection  rationale. 
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b.  Identification  of  the  equipment  to  be  tested 
(with  identification  of  the  computer  programs 
to  be  used  for  the  test,  if  applicable)  and  the 
number  of  test  items  of  each  equipment. 

c.  Test  duration  and  the  appropriate  test  plan  and 
test  environments.  The  test  plan  and  test 
environments  (if  life/mission  profiles  are  not 
specified  by  the  PA)  shall  be  derived  from 
MIL-STD-781.  If  it  is  deemed  that 
alternative  procedures  are  more  appropriate, 
prior  PA  approval  shall  be  requested  with 
sufficient  selection  rationale  to  permit 
procuring  activity  evaluation, 

d.  A  test  schedule  that  is  reasonable  and  feasible, 
permits  testing  of  equipment  which  are  represen¬ 
tative  of  the  approved  production  configuration, 
and  allows  sufficient  time,  as  specified  in  the 
contract,  for  PA  review  and  approval  of  each  test 
procedure  and  test  setup. 

303.2.3  Detailed  test  procedures  shall  be  prepared  for  the 
tests  that  are  included  in  the  RQT  plan. 
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and  test  procedures  shall  be  submitted  to  the 
procuring  activity  for  its  review  and  approval. 
These  documents,  as  approved,  shall  be  incorporated 
into  the  contract  and  shall  become  the  basis  for 
contractual  compliance. 


TASK  304 


304.1 


304.2 

304.2 

304.2 


PRODUCTION  RELIABILITY  ACCEPTANCE  TEST  (PRAT)  PROGRAM 

PURPOSE.  The  purpose  of  Task  304  is  to  assure  that  the 
reliability  of  the  hardware  is  not  degraded  as  the 
result  of  changes  in  tooling,  processes,  work  flow, 
design,  parts  quality,  or  other  characteristics 
identified  by  the  PA. 

TASK  DESCRIPTION 

.1  Production  reliability  acceptance  testing  shall  be 
conducted  on  production  equipments  which  shall  be 
identified  by  the  procuring  activity, 

.2  A  PRAT1 plan  shall  be  prepared  in  accordance  with  the 
requirements  of  MIL-STD-781 ,  or  alternative  approved 
by  the  PA,  and  shall  include  the  following,  subject  to 
PA  approval  prior  to  initiation  of  testing: 

a.  Teat  objectives  and  selection  rationale. 

b.  Identification  of  the  equipment  to  be  tested  and 
the  number  of  test  samples  of  each  equipment. 

c.  Test  duration,  test  frequency,  and  the  appropriate 
teat  plan  and  test  environments.  The  test  plan 
and  teat  environments  (if  mission  profiles  are  not 
specified  by  the  PA)  shall  be  derived  from 
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304.2.3 


304.2.4 


MIL-STD-781.  If  it  is  deemed  that  alternative 
procedures  are  more  appropriate,  prior  PA  approval 
shall  be  requested  with  sufficient  selection 
rationale  to  permit  procuring  activity  evaluation, 
d.  A  test  schedule  that  is  reasonable  and  feasible, 
and  in  consonance  with  the  production  delivery 
schedule. 

Detailed  test  procedures  shall  be  prepared  for  the 
tests  that  are  included  in  the  PRAT  plan  or  the 
equipment  specification. 

As  specified  by  the  procuring  activity,  the  PRAT  plan 
and  procedures  shall  be  submitted  to  the  procuring 
activity  for  its  review  and  approval.  These  documents, 
as  approved  by  the  procuring  activity,  shall  be  in¬ 
corporated  into  the  contract  and  shall  become  the 
basis  for  contractual  compliance. 
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MISSION 

of 

Rome  Air  Development  Center 

RAflC  plant,  and  exec utes  research,  development,  test  and 
selected  acquisition  pn.ogn.am  in  support  ofi  Command,  Control 
Communications  and  Intelligence  |C3I)  activities.  Technical 
and  enginee/Ung  suppont  uiithin  areas  oi  technical  competence 
is  provided  to  ESQ  Program  O^ices  (Pfl.il  and  other  ESD 
elements.  The  principal  technical  mission  areas  one 
communications,  electromagnetic  guidance  and  control,  sur¬ 
veillance  o £  ground  and  aerospace  objects,  intelligence  data 
collection  and  handling,  infiormxtion  system  technology, 
ionospheric  propagation,  solid  state  sciences,  microwave 
physics  and  electronic  reliability,  maintainability  and 
compatibility. 
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